ANTLR3_LEXER_struct - Man Page

void * ctx
A generated lexer has an mTokens() function, which needs the context pointer of the generated lexer, not the base lexer interface this is stored here and initialized by the generated code (or manually if this is a manually built lexer.
pANTLR3_COMMON_TOKEN(* emit )(struct ANTLR3_LEXER_struct *lexer)
Pointer to a function that constructs a new token from the lexer stored information.
void(* emitNew )(struct ANTLR3_LEXER_struct *lexer, pANTLR3_COMMON_TOKEN token)
Pointer to a function that emits the supplied token as the next token in the stream.
void(* free )(struct ANTLR3_LEXER_struct *lexer)
Pointer to a function that knows how to free the resources of a lexer.
ANTLR3_MARKER(* getCharIndex )(struct ANTLR3_LEXER_struct *lexer)
ANTLR3_UINT32(* getCharPositionInLine )(struct ANTLR3_LEXER_struct *lexer)
ANTLR3_UINT32(* getLine )(struct ANTLR3_LEXER_struct *lexer)
Pointer to function to return the current line number in the input stream.
pANTLR3_STRING(* getText )(struct ANTLR3_LEXER_struct *lexer)
Pointer to function to return the text so far for the current token being generated.
pANTLR3_INPUT_STREAM input
A pointer to the character stream whence this lexer is receiving characters.
void(* matchAny )(struct ANTLR3_LEXER_struct *lexer)
Pointer to a function that matches the next token/char in the input stream regardless of what it actaully is.
ANTLR3_BOOLEAN(* matchc )(struct ANTLR3_LEXER_struct *lexer, ANTLR3_UCHAR c)
Pointer to a function that matches and consumes the specified character from the input stream.
ANTLR3_BOOLEAN(* matchRange )(struct ANTLR3_LEXER_struct *lexer, ANTLR3_UCHAR low, ANTLR3_UCHAR high)
Pointer to a function that matches any character in the supplied range (I suppose it could be a token range too but this would only be useful if the tokens were in tsome guaranteed order which is only going to happen with a hand crafted token set).
ANTLR3_BOOLEAN(* matchs )(struct ANTLR3_LEXER_struct *lexer, ANTLR3_UCHAR *string)
Pointer to a function that attempts to match and consume the specified string from the input stream.
void(* mTokens )(void *ctx)
Pointer to the user provided (either manually or through code generation function that causes the lexer rules to run the lexing rules and produce the next token if there iss one.
void(* popCharStream )(struct ANTLR3_LEXER_struct *lexer)
Pointer to a function that abandons the current input stream, whether it is empty or not and reverts to the previous stacked input stream.
void(* pushCharStream )(struct ANTLR3_LEXER_struct *lexer, pANTLR3_INPUT_STREAM input)
Pointer to a function that switches the current character input stream to a new one, saving the old one, which we will revert to at the end of this new one.
pANTLR3_BASE_RECOGNIZER rec
Pointer to the implementation of a base recognizer, which the lexer creates and then overrides with its own lexer oriented functions (the default implementation is parser oriented).
void(* recover )(struct ANTLR3_LEXER_struct *lexer)
Pointer to a function that recovers from an error found in the input stream.
void(* setCharStream )(struct ANTLR3_LEXER_struct *lexer, pANTLR3_INPUT_STREAM input)
Pointer to a function that sets the charstream source for the lexer and causes it to be reset.
void * super
If there is a super structure that is implementing the lexer, then a pointer to it can be stored here in case implementing functions are overridden by this super structure.

A generated lexer has an mTokens() function, which needs the context pointer of the generated lexer, not the base lexer interface this is stored here and initialized by the generated code (or manually if this is a manually built lexer.

Referenced by nextTokenStr().

Pointer to a function that constructs a new token from the lexer stored information.

Referenced by antlr3LexerNew().

Pointer to a function that emits the supplied token as the next token in the stream.

Referenced by antlr3LexerNew().

Pointer to a function that knows how to free the resources of a lexer.

Referenced by antlr3LexerNew().

Referenced by antlr3LexerNew(), emit(), getText(), and nextTokenStr().

Referenced by antlr3LexerNew().

Pointer to function to return the current line number in the input stream.

Referenced by antlr3LexerNew(), and nextTokenStr().

Pointer to function to return the text so far for the current token being generated.

Referenced by antlr3LexerNew().

A pointer to the character stream whence this lexer is receiving characters. TODO: I may come back to this and implement charstream outside the input stream as per the java implementation.

Referenced by alreadyParsedRule(), antlr3RecognitionExceptionNew(), displayRecognitionError(), emit(), getCharIndex(), getCharPositionInLine(), getLine(), getText(), matchAny(), matchc(), matchRange(), matchs(), memoize(), nextTokenStr(), popCharStream(), pushCharStream(), recover(), and setCharStream().

Pointer to a function that matches the next token/char in the input stream regardless of what it actaully is.

Referenced by antlr3LexerNew().

Pointer to a function that matches and consumes the specified character from the input stream. The input stream is required to provide characters via LA() as UTF32 characters. The default lexer implementation is source encoding agnostic and so input streams do not generally need to override the default implmentation.

Referenced by antlr3LexerNew().

Pointer to a function that matches any character in the supplied range (I suppose it could be a token range too but this would only be useful if the tokens were in tsome guaranteed order which is only going to happen with a hand crafted token set).

Referenced by antlr3LexerNew().

Pointer to a function that attempts to match and consume the specified string from the input stream. Note that strings muse be passed as terminated arrays of ANTLR3_UCHAR. Strings are terminated with 0xFFFFFFFF, which is an invalid UTF32 character

Referenced by antlr3LexerNew().

Pointer to the user provided (either manually or through code generation function that causes the lexer rules to run the lexing rules and produce the next token if there iss one. This is called from nextToken() in the pANTLR3_TOKEN_SOURCE. Note that the input parameter for this funciton is the generated lexer context (stored in ctx in this interface) it is a generated function and expects the context to be the generated lexer.

Referenced by antlr3LexerNew(), and nextTokenStr().

Pointer to a function that abandons the current input stream, whether it is empty or not and reverts to the previous stacked input stream.

Referenced by antlr3LexerNew(), and nextToken().

Pointer to a function that switches the current character input stream to a new one, saving the old one, which we will revert to at the end of this new one.

Referenced by antlr3LexerNew().

Pointer to the implementation of a base recognizer, which the lexer creates and then overrides with its own lexer oriented functions (the default implementation is parser oriented). This also contains a token source interface, which the lexer instance will provide to anything that needs it, which is anything else that implements a base recognizer, such as a parser.

Referenced by antlr3LexerNew(), displayRecognitionError(), emit(), emitNew(), freeLexer(), getText(), matchc(), matchRange(), matchs(), nextToken(), nextTokenStr(), popCharStream(), pushCharStream(), reset(), and setCharStream().

Pointer to a function that recovers from an error found in the input stream. Generally, this will be a #ANTLR3_EXCEPTION_NOVIABLE_ALT but it could also be from a mismatched token that the (*match)() could not recover from.

Referenced by antlr3LexerNew(), matchc(), matchRange(), matchs(), and nextTokenStr().

Pointer to a function that sets the charstream source for the lexer and causes it to be reset.

Referenced by antlr3LexerNew(), popCharStream(), and pushCharStream().

If there is a super structure that is implementing the lexer, then a pointer to it can be stored here in case implementing functions are overridden by this super structure.