NAME

SYNOPSIS

EXPERIMENTAL API

TokenBatch's API should be considered experimental and is likely to change.

DESCRIPTION

A TokenBatch is a collection of Tokens which you can add to, then iterate over.

METHODS

new

my $batch = KinoSearch1::Analysis::TokenBatch->new;

Constructor.

append

$batch->append( $text, $start_offset, $end_offset, $pos_inc );

Add a Token to the end of the batch. Accepts either three or four arguments: text, start_offset, end_offset, and an optional position increment which defaults to 1 if not supplied. For a description of what these arguments mean, see the docs for Token.

next

while ( $batch->next ) {
# ...
}

Proceed to the next token in the TokenBatch. Returns true if the TokenBatch ends up located at valid token.

ACCESSOR METHODS

All of TokenBatch's accessor methods affect the current Token. Calling any of these methods when the TokenBatch is not located at a valid Token will trigger an exception.