java.io
Class StreamTokenizer

java.lang.Object
  extended by java.io.StreamTokenizer

public class StreamTokenizer
extends Object

This class parses streams of characters into tokens. There are a million-zillion flags that can be set to control the parsing, as described under the various method headings.


Field Summary
 double nval
          The numeric value associated with number tokens.
 String sval
          The String associated with word and string tokens.
static int TT_EOF
          A constant indicating that the end of the stream has been read.
static int TT_EOL
          A constant indicating that the end of the line has been read.
static int TT_NUMBER
          A constant indicating that a number token has been read.
static int TT_WORD
          A constant indicating that a word token has been read.
 int ttype
          Contains the type of the token read resulting from a call to nextToken The rules are as follows: For a token consisting of a single ordinary character, this is the value of that character.
 
Constructor Summary
StreamTokenizer(InputStream is)
          Deprecated. Since JDK 1.1.
StreamTokenizer(Reader r)
          This method initializes a new StreamTokenizer to read characters from a Reader and parse them.
 
Method Summary
 void commentChar(int ch)
          This method sets the comment attribute on the specified character.
 void eolIsSignificant(boolean flag)
          This method sets a flag that indicates whether or not the end of line sequence terminates and is a token.
 int lineno()
          This method returns the current line number.
 void lowerCaseMode(boolean flag)
          This method sets a flag that indicates whether or not alphabetic tokens that are returned should be converted to lower case.
 int nextToken()
          This method reads the next token from the stream.
 void ordinaryChar(int ch)
          This method makes the specified character an ordinary character.
 void ordinaryChars(int low, int hi)
          This method makes all the characters in the specified range, range terminators included, ordinary.
 void parseNumbers()
          This method sets the numeric attribute on the characters '0' - '9' and the characters '.' and '-'.
 void pushBack()
          Puts the current token back into the StreamTokenizer so nextToken will return the same value on the next call.
 void quoteChar(int ch)
          This method sets the quote attribute on the specified character.
 void resetSyntax()
          This method removes all attributes (whitespace, alphabetic, numeric, quote, and comment) from all characters.
 void slashSlashComments(boolean flag)
          This method sets a flag that indicates whether or not "C++" language style comments ("//" comments through EOL ) are handled by the parser.
 void slashStarComments(boolean flag)
          This method sets a flag that indicates whether or not "C" language style comments (with nesting not allowed) are handled by the parser.
 String toString()
          This method returns the current token value as a String in the form "Token[x], line n", where 'n' is the current line numbers and 'x' is determined as follows.
 void whitespaceChars(int low, int hi)
          This method sets the whitespace attribute for all characters in the specified range, range terminators included.
 void wordChars(int low, int hi)
          This method sets the alphabetic attribute for all characters in the specified range, range terminators included.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

TT_EOF

public static final int TT_EOF
A constant indicating that the end of the stream has been read.

See Also:
Constant Field Values

TT_EOL

public static final int TT_EOL
A constant indicating that the end of the line has been read.

See Also:
Constant Field Values

TT_NUMBER

public static final int TT_NUMBER
A constant indicating that a number token has been read.

See Also:
Constant Field Values

TT_WORD

public static final int TT_WORD
A constant indicating that a word token has been read.

See Also:
Constant Field Values

ttype

public int ttype
Contains the type of the token read resulting from a call to nextToken The rules are as follows:


sval

public String sval
The String associated with word and string tokens.


nval

public double nval
The numeric value associated with number tokens.

Constructor Detail

StreamTokenizer

public StreamTokenizer(InputStream is)
Deprecated. Since JDK 1.1.

This method reads bytes from an InputStream and tokenizes them. For details on how this method operates by default, see StreamTokenizer(Reader).

Parameters:
is - The InputStream to read from

StreamTokenizer

public StreamTokenizer(Reader r)
This method initializes a new StreamTokenizer to read characters from a Reader and parse them. The char values have their hight bits masked so that the value is treated a character in the range of 0x0000 to 0x00FF.

This constructor sets up the parsing table to parse the stream in the following manner:

Parameters:
r - The Reader to read chars from
Method Detail

commentChar

public void commentChar(int ch)
This method sets the comment attribute on the specified character. Other attributes for the character are cleared.

Parameters:
ch - The character to set the comment attribute for, passed as an int

eolIsSignificant

public void eolIsSignificant(boolean flag)
This method sets a flag that indicates whether or not the end of line sequence terminates and is a token. The defaults to false

Parameters:
flag - true if EOF is significant, false otherwise

lineno

public int lineno()
This method returns the current line number. Note that if the pushBack() method is called, it has no effect on the line number returned by this method.

Returns:
The current line number

lowerCaseMode

public void lowerCaseMode(boolean flag)
This method sets a flag that indicates whether or not alphabetic tokens that are returned should be converted to lower case.

Parameters:
flag - true to convert to lower case, false otherwise

nextToken

public int nextToken()
              throws IOException
This method reads the next token from the stream. It sets the ttype variable to the appropriate token type and returns it. It also can set sval or nval as described below. The parsing strategy is as follows:

Returns:
The token type
Throws:
IOException - If an I/O error occurs

ordinaryChar

public void ordinaryChar(int ch)
This method makes the specified character an ordinary character. This means that none of the attributes (whitespace, alphabetic, numeric, quote, or comment) will be set on this character. This character will parse as its own token.

Parameters:
ch - The character to make ordinary, passed as an int

ordinaryChars

public void ordinaryChars(int low,
                          int hi)
This method makes all the characters in the specified range, range terminators included, ordinary. This means the none of the attributes (whitespace, alphabetic, numeric, quote, or comment) will be set on any of the characters in the range. This makes each character in this range parse as its own token.

Parameters:
low - The low end of the range of values to set the whitespace attribute for
hi - The high end of the range of values to set the whitespace attribute for

parseNumbers

public void parseNumbers()
This method sets the numeric attribute on the characters '0' - '9' and the characters '.' and '-'. When this method is used, the result of giving other attributes (whitespace, quote, or comment) to the numeric characters may vary depending on the implementation. For example, if parseNumbers() and then whitespaceChars('1', '1') are called, this implementation reads "121" as 2, while some other implementation will read it as 21.


pushBack

public void pushBack()
Puts the current token back into the StreamTokenizer so nextToken will return the same value on the next call. May cause the lineno method to return an incorrect value if lineno is called before the next call to nextToken.


quoteChar

public void quoteChar(int ch)
This method sets the quote attribute on the specified character. Other attributes for the character are cleared.

Parameters:
ch - The character to set the quote attribute for, passed as an int.

resetSyntax

public void resetSyntax()
This method removes all attributes (whitespace, alphabetic, numeric, quote, and comment) from all characters. It is equivalent to calling ordinaryChars(0x00, 0xFF).

See Also:
ordinaryChars(int, int)

slashSlashComments

public void slashSlashComments(boolean flag)
This method sets a flag that indicates whether or not "C++" language style comments ("//" comments through EOL ) are handled by the parser. If this is true commented out sequences are skipped and ignored by the parser. This defaults to false.

Parameters:
flag - true to recognized and handle "C++" style comments, false otherwise

slashStarComments

public void slashStarComments(boolean flag)
This method sets a flag that indicates whether or not "C" language style comments (with nesting not allowed) are handled by the parser. If this is true commented out sequences are skipped and ignored by the parser. This defaults to false.

Parameters:
flag - true to recognized and handle "C" style comments, false otherwise

toString

public String toString()
This method returns the current token value as a String in the form "Token[x], line n", where 'n' is the current line numbers and 'x' is determined as follows.

Overrides:
toString in class Object
Returns:
the String representing this Object, which may be null
See Also:
Object.getClass(), Object.hashCode(), Class.getName(), Integer.toHexString(int)

whitespaceChars

public void whitespaceChars(int low,
                            int hi)
This method sets the whitespace attribute for all characters in the specified range, range terminators included.

Parameters:
low - The low end of the range of values to set the whitespace attribute for
hi - The high end of the range of values to set the whitespace attribute for

wordChars

public void wordChars(int low,
                      int hi)
This method sets the alphabetic attribute for all characters in the specified range, range terminators included.

Parameters:
low - The low end of the range of values to set the alphabetic attribute for
hi - The high end of the range of values to set the alphabetic attribute for