Do nothing - TS is positioned before the first token automatically by default.

Token sequence will always be positioned between tokens
when using one of the operations above
(token() will return null to signal between-tokens location).

Start iterating through the tokens in forward/backward direction
by using moveNext() or movePrevious().
If moveNext() or movePrevious() returned
true then TS is positioned
over a concrete token retrievable by token().
Its offset can be retrieved by offset().

This object should be used by a single thread only. For token hierarchies
over mutable input sources the obtaining and using of the token sequence
needs to be done under a read-lock of the input source.

Get embedded token sequence that possibly joins multiple embeddings
with the same language paths (if the embeddings allow it - see
LanguageEmbedding.joinSections()) into a single input text
which is then lexed as a single continuous text.

The lifetime of the returned token instance may be limited for mutable inputs.
The token instance should not be held across the input source modifications.

Returns:

token instance to which this token sequence is currently positioned
or null if this token sequence is not positioned to any token which may
happen after TS creation or after use of move(int) or moveIndex(int).

offsetToken

Similar to token() but always returns a non-flyweight token
with the appropriate offset.
If the current token is flyweight then this method replaces it
with the corresponding non-flyweight token which it then returns.
Subsequent calls to token() will also return this non-flyweight token.

This method may be handy if the token instance is referenced in a standalone way
(e.g. in an expression node of a parse tree) and it's necessary
to get the appropriate offset from the token itself
later when a token sequence will not be available.

offset

public int offset()

Get the offset of the current token in the underlying input.
The token's offset should never be computed by a client of the token sequence
by adding/subtracting tokens' length to a client's variable because
in case of the immutable token sequences there can be gaps
between tokens if some tokens get filtered out.
Instead this method should always be used because it offers
best performance with a constant time complexity.

embedded

Get embedded token sequence if the token
to which this token sequence is currently positioned
has a language embedding.
If there is a custom embedding created by
createEmbedding(Language,int,int) it will be returned
instead of the default embedding
(the one created by LanguageHierarchy.embedding()
or LanguageProvider).

embeddedJoined

Get embedded token sequence that possibly joins multiple embeddings
with the same language paths (if the embeddings allow it - see
LanguageEmbedding.joinSections()) into a single input text
which is then lexed as a single continuous text.
If any of the resulting tokens crosses embedding's boundaries then the token
is split into multiple part tokens.
If the embedding does not join sections then this method behaves
like embedded().

Returns:

embedded sequence or null if no embedding exists for this token.
The token sequence will be positioned before first token of this embedding
or to a join token in case the first token of this embedding is part of the join token.

createEmbedding

Create language embedding described by the given parameters.
If the underying text input is mutable then this method should only be called
within a write lock over the text input.

Parameters:

embeddedLanguage - non-null embedded language

startSkipLength - >=0 number of characters in an initial part of the token
for which the language embedding is defined that should be excluded
from the embedded section. The excluded characters will not be lexed
and there will be no tokens created for them.

endSkipLength - >=0 number of characters at the end of the token
for which the language embedding is defined that should be excluded
from the embedded section. The excluded characters will not be lexed
and there will be no tokens created for them.

joinSections - whether sections with this embedding should be joined
across the input source or whether they should stay separate.
For example for HTML sections embedded in JSP this flag should be true:

Only the embedded sections with the same language path can be joined.
If preceding embeddings requested sections joining for the particular language path
then this parameter will be updated from false to true automatically by the method.

Returns:

true if the embedding was created successfully or false if an embedding
with the given language already exists for this token.

Subsequent moveNext() or movePrevious() is needed to fetch
a concrete token.
If the offset is too big then the token sequence will be positioned
behind the last token.

If token filtering is used there may be gaps that are not covered
by any tokens and if the offset is contained in such gap then
the token sequence will be positioned before the token that precedes the gap.

Parameters:

offset - absolute offset to which the token sequence should be moved.

Returns:

difference between the reqeuested offset
and the start offset of the token
before which the the token sequence gets positioned.
If positioned right after the last token then (offset - last-token-end-offset)
is returned.