lucene-dev mailing list archives

TestCase for KeywordAnalyzer split into KeywordTokenizer/KeywordAnalyzer

Date

Fri, 11 Feb 2005 13:47:19 GMT

Oops, attached wrong file...
This testcase fails on the original KeywordAnalyzer and passes with the
new Analyzer/Tokenizer pair.
The tests still fail on org.apache.lucene.analysis.ru.TestRussianStem
Anybody checking this?
Luc
______________________________________________
From: Vanlerberghe, Luc
Sent: vrijdag 11 februari 2005 13:25
To: lucene-dev@jakarta.apache.org
Subject: KeywordAnalyzer split into
KeywordTokenizer/KeywordAnalyzer
Hi all,
I found Erik's KeywordAnalyzer very useful (I had just written a similar
but more limited one a few hours before him) but I wanted a
KeywordTokenizer that I would then be able to use in different
circumstances more easily (E.g. chain it to a LowercaseFilter)
So I took the liberty to modify his code into a KeywordTokenizer and let
the KeywordAnalyzer return an instance of it.
It also solves the problem that the original KeywordAnalyzer never
closed its Reader (TokenStream.close() was called implicitly, but that
has an empty implementation)
What is the proper way to sumbit this?
I attached a diff that should be applied in
contrib/analyzers/src/java/org/apache/lucene/analysis
Should I submit it as an attachment to a Bugzilla report instead?
Luc