Re: Does StreamTokenizer.pushBack correctly update TT_EOF ?
In article
<e458e6e8-7543-4913-bf9f-865022e65ba3@r15g2000prh.googlegroups.com>,
enokacorea@gmail.com wrote:
[...]
I have to say this still seems counterintuitive. What's left to do after
seeing TT_EOF?
[...]
You see, if I push back something near the end of a loop I want it
processed at the start of the next loop. If TT_EOF is not set
correctly the last token will not be processed correctly.
This is what I tried to resolve with the logic I described earlier.
I am still having my original problem though !
Trying to debug this I tried a small test and found that - nextToken()
immediately followed by pushBack() does not give the same result as
not having both statements !!
Any thoughts on this ?
Yes, the effect of pushBack() is limited to giving the same result from
nextToken(); nval and sval are unchanged. The mechanism seems intended
to provide inexpensive, one-token look-ahead, rather than unlimited undo.
<code>
import java.io.*;
public class Tokenizer {
public static void main(String[] args) throws IOException {
StreamTokenizer tokens = new
StreamTokenizer(new StringReader("Test"));
int token;
token = tokens.nextToken();
print(tokens, token);
token = tokens.nextToken();
print(tokens, token);
tokens.pushBack();
token = tokens.nextToken();
print(tokens, token);
}
private static void print(StreamTokenizer tokens, int token) {
System.out.println((token == StreamTokenizer.TT_EOF)
+ " " + tokens.sval);
}
}
</code>
<console>
false Test
true null
true null
</console>
Here's a more elaborate example:
<http://sites.google.com/site/drjohnbmatthews/enumerated-functions>
Notice how encountering TT_EOF (defined as Symbol.END) signals
successful parsing; any other result represents a (not otherwise
identified) syntax error.
--
John B. Matthews
trashgod at gmail dot com
home dot woh dot rr dot com slash jbmatthews