Text::Categorize::Textrank provides a routine for ranking the words in text as potential keywords. It implements a version of the textrank algorithm from the report TextRank: Bringing Order into Texts by R. Mihalcea and P. Tarau.

The routine getTextrankOfListOfTokens returns a hash reference containing the textrank value for all the tokens in the lists provided; the textrank values sum to one. The textrank of a token is its pagerank in the graph obtained by joining neighboring tokens with an edge, called the token graph.

Usually, listOfTokens should not be applied to all the words of the text. The complete textrank algorithm first filters the words in the text to only the nouns and adjectives. See Text::Categorize::Textrank::En to compute the textrank of English text.

listOfTokens is an array reference containing the list of tokens that are to be ranked using textrank. Each list is also an array reference of tokens that should correspond to the list of tokens in a sentence. For example, [[qw(This is the first sentence)], [qw(Here is the second sentence)]].

For each token in the listOfTokens, edgeCreationSpan is the number of successive tokens used to make an edge in the token graph. For example, if edgeCreationSpan is two, then given the token sequence "apple orange pear" the edges [apple, orange] and [apple, pear] will be added to the token graph for the token apple. The default is one.

Note that loop edges are ignored. For example, if edgeCreationSpan is two, then given the token sequence "daba daba doo" the edge [daba, daba] is disguarded but the edge [daba, doo] is added to the token graph.

If addEdgesSpanningLists is true, then when building the token graph, links between the tokens at the end of a list and the beginning of the next list will be made. For example, for the lists [[qw(This is the first list)], [qw(Here is the second list)]] the edge [list, Here] will be added to the token graph. The default is true.

tokenWeights is an optional hash reference that can provide a weight for a subset of the tokens provided by listOfTokens. If tokenWeights is not defined for any token in listOfTokens, then each token has a weight of one. If tokenWeights is defined for at least one node in the graph, then the default weight of any undefined token is zero.