Class LexerActionExecutor

Represents an executor for a sequence of lexer actions which traversed during
the matching operation of a lexer rule (token).

The executor tracks position information for position-dependent lexer actions
efficiently, ensuring that actions appearing only at the end of the rule do
not cause bloating of the DFA created for the lexer.

Method Detail

append

Creates a LexerActionExecutor which executes the actions for
the input lexerActionExecutor followed by a specified
lexerAction.

Parameters:

lexerActionExecutor - The executor for actions already traversed by
the lexer while matching a token within a particular
LexerATNConfig. If this is null, the method behaves as
though it were an empty executor.

lexerAction - The lexer action to execute after the actions
specified in lexerActionExecutor.

Returns:

A LexerActionExecutor for executing the combine actions
of lexerActionExecutor and lexerAction.

fixOffsetBeforeMatch

Creates a LexerActionExecutor which encodes the current offset
for position-dependent lexer actions.

Normally, when the executor encounters lexer actions where
LexerAction.isPositionDependent() returns true, it calls
IntStream.seek(int) on the input CharStream to set the input
position to the end of the current token. This behavior provides
for efficient DFA representation of lexer actions which appear at the end
of a lexer rule, even when the lexer rule matches a variable number of
characters.

Prior to traversing a match transition in the ATN, the current offset
from the token start index is assigned to all position-dependent lexer
actions which have not already been assigned a fixed offset. By storing
the offsets relative to the token start index, the DFA representation of
lexer actions which appear in the middle of tokens remains efficient due
to sharing among tokens of the same length, regardless of their absolute
position in the input stream.

If the current executor already has offsets assigned to all
position-dependent lexer actions, the method returns this.

Parameters:

offset - The current offset to assign to all position-dependent
lexer actions which do not already have offsets assigned.

Returns:

A LexerActionExecutor which stores input stream offsets
for all position-dependent lexer actions.

input - The input stream which is the source for the current token.
When this method is called, the current IntStream.index() for
input should be the start of the following token, i.e. 1
character past the end of the current token.

startIndex - The token start index. This value may be passed to
IntStream.seek(int) to set the input position to the beginning
of the token.