ANTLR3_TOKEN_STREAM_struct - Man Page

pANTLR3_COMMON_TOKEN(* _LT )(struct ANTLR3_TOKEN_STREAM_struct *tokenStream, ANTLR3_INT32 k)
Get Token at current input pointer + i ahead where i=1 is next Token.
pANTLR3_DEBUG_EVENT_LISTENER debugger
Debugger interface, is this is a debugging token stream.
void(* free )(struct ANTLR3_TOKEN_STREAM_struct *tokenStream)
Function that knows how to free the memory for an ANTLR3_TOKEN_STREAM.
pANTLR3_COMMON_TOKEN(* get )(struct ANTLR3_TOKEN_STREAM_struct *tokenStream, ANTLR3_UINT32 i)
Get a token at an absolute index i; 0..n-1.
pANTLR3_TOKEN_SOURCE(* getTokenSource )(struct ANTLR3_TOKEN_STREAM_struct *tokenStream)
Where is this stream pulling tokens from? This is not the name, but a pointer into an interface that contains a ANTLR3_TOKEN_SOURCE interface.
ANTLR3_BOOLEAN initialStreamState
Indicates the initial stream state for dbgConsume()
pANTLR3_INT_STREAM istream
All input streams implement the ANTLR3_INT_STREAM interface...
void(* setDebugListener )(struct ANTLR3_TOKEN_STREAM_struct *tokenStream, pANTLR3_DEBUG_EVENT_LISTENER debugger)
Function that sets the token stream into debugging mode.
void(* setTokenSource )(struct ANTLR3_TOKEN_STREAM_struct *tokenStream, pANTLR3_TOKEN_SOURCE tokenSource)
Function that installs a token source for teh stream.
void * super
Whatever is providing this interface needs a pointer to itself so that this can be passed back to it whenever the api functions are called.
pANTLR3_TOKEN_SOURCE tokenSource
Pointer to the token source for this stream.
pANTLR3_STRING(* toString )(struct ANTLR3_TOKEN_STREAM_struct *tokenStream)
Return the text of all the tokens in the stream, as the old tramp in Leeds market used to say; 'Get the lot!'.
pANTLR3_STRING(* toStringSS )(struct ANTLR3_TOKEN_STREAM_struct *tokenStream, ANTLR3_UINT32 start, ANTLR3_UINT32 stop)
Return the text of all tokens from start to stop, inclusive.
pANTLR3_STRING(* toStringTT )(struct ANTLR3_TOKEN_STREAM_struct *tokenStream, pANTLR3_COMMON_TOKEN start, pANTLR3_COMMON_TOKEN stop)
Because the user is not required to use a token with an index stored in it, we must provide a means for two token objects themselves to indicate the start/end location.

Get Token at current input pointer + i ahead where i=1 is next Token. i<0 indicates tokens in the past. So -1 is previous token and -2 is two tokens ago. LT(0) is undefined. For i>=n, return Token.EOFToken. Return null for LT(0) and any index that results in an absolute address that is negative.

Referenced by _LA(), antlr3CommonTokenDebugStreamSourceNew(), antlr3CommonTokenStreamNew(), antlr3RecognitionExceptionNew(), dbgConsume(), getMissingSymbol(), and setDebugListener().

Debugger interface, is this is a debugging token stream.

Referenced by antlr3CommonTokenDebugStreamSourceNew(), consumeInitialHiddenTokens(), dbgConsume(), dbgLA(), dbgMark(), dbgRewindLast(), dbgRewindStream(), and setDebugListener().

Function that knows how to free the memory for an ANTLR3_TOKEN_STREAM.

Referenced by antlr3CTSFree(), and antlr3TokenStreamNew().

Get a token at an absolute index i; 0..n-1. This is really only needed for profiling and debugging and token stream rewriting. If you don't want to buffer up tokens, then this method makes no sense for you. Naturally you can't use the rewrite stream feature. I believe DebugTokenStream can easily be altered to not use this method, removing the dependency.

Referenced by antlr3CommonTokenStreamNew(), consumeInitialHiddenTokens(), dbgConsume(), getTokensSet(), and toStringSS().

Where is this stream pulling tokens from? This is not the name, but a pointer into an interface that contains a ANTLR3_TOKEN_SOURCE interface. The Token Source interface contains a pointer to the input stream and a pointer to a function that returns the next token.

Referenced by antlr3CommonTokenStreamNew(), and toStringSS().

Indicates the initial stream state for dbgConsume()

Referenced by antlr3CommonTokenDebugStreamSourceNew(), consumeInitialHiddenTokens(), dbgConsume(), dbgLA(), dbgTokLT(), and setDebugListener().

All input streams implement the ANTLR3_INT_STREAM interface...

Referenced by alreadyParsedRule(), antlr3CommonTokenDebugStreamSourceNew(), antlr3CommonTokenStreamNew(), antlr3CTSFree(), antlr3RecognitionExceptionNew(), consumeUntil(), consumeUntilSet(), dbgTokLT(), displayRecognitionError(), fillBuffer(), getTokensSet(), match(), matchAny(), memoize(), mismatch(), recover(), recoverFromMismatchedElement(), recoverFromMismatchedSet(), recoverFromMismatchedToken(), setDebugListener(), skipOffTokenChannels(), synpred(), tokLT(), toString(), and toStringSS().

Function that sets the token stream into debugging mode.

Referenced by antlr3CommonTokenStreamNew(), and setDebugListener().

Function that installs a token source for teh stream.

Referenced by antlr3CommonTokenStreamNew(), and antlr3CommonTokenStreamSourceNew().

Whatever is providing this interface needs a pointer to itself so that this can be passed back to it whenever the api functions are called.

Referenced by antlr3CommonTokenStreamNew(), antlr3CTSFree(), antlr3RecognitionExceptionNew(), consume(), get(), getMissingSymbol(), seek(), size(), tindex(), tokLT(), toString(), and toStringSS().

Pointer to the token source for this stream.

Referenced by fillBuffer(), getTokenSource(), setTokenSource(), and tokLT().

Return the text of all the tokens in the stream, as the old tramp in Leeds market used to say; 'Get the lot!'.

Referenced by antlr3CommonTokenStreamNew().

Return the text of all tokens from start to stop, inclusive. If the stream does not buffer all the tokens then it can just return an empty ANTLR3_STRING or NULL; Grammars should not access $ruleLabel.text in an action in that case.

Referenced by antlr3CommonTokenStreamNew(), toString(), and toStringTT().

Because the user is not required to use a token with an index stored in it, we must provide a means for two token objects themselves to indicate the start/end location. Most often this will just delegate to the other toString(int,int). This is also parallel with the pTREENODE_STREAM->toString(Object,Object).

Referenced by antlr3CommonTokenStreamNew().