Represented as a coupled graph of class instances, this
all-in-memory index store implementation delivers search
results up to a 100 times faster than the file-centric RAMDirectory
at the cost of greater RAM consumption.

Package org.apache.lucene.store.instantiated Description

InstantiatedIndex, alternative RAM store for small corpora.

WARNING: This contrib is experimental and the APIs may change without warning.

Abstract

Represented as a coupled graph of class instances, this
all-in-memory index store implementation delivers search
results up to a 100 times faster than the file-centric RAMDirectory
at the cost of greater RAM consumption.

API

Just as the default store implementation, InstantiatedIndex
comes with an IndexReader and IndexWriter. The latter share
many method signatures with the file-centric IndexWriter.

It is also possible to load the content of another index
by passing an IndexReader to the InstantiatedIndex constructor.

Performance

At a few thousand ~160 characters long documents
InstantiatedIndex outperforms RAMDirectory some 50x,
15x at 100 documents of 2000 characters length,
and is linear to RAMDirectory at 10,000 documents of 2000 characters length.

Mileage may vary depending on term saturation.

Populated with a single document InstantiatedIndex is almost, but not quite, as fast as MemoryIndex.

It takes more or less the same time to populate an InstantiatedIndex
as it takes to populate a RAMDirectory. Hardly any effort has been put
in to optimizing the InstantiatedIndexWriter, only minimizing the amount
of time needed to write-lock the index has been considered.

Caveats

No locks! Consider using InstantiatedIndex as if it was immutable.

No documents with fields containing readers.

No field selection when retrieving documents, as all stored field are available in memory.

Any document returned must cloned if they are to be touched.

Norms array returned must not be touched.

Use cases

Could replace any small index that could do with greater response time.
spell check a priori index,
the index of new documents exposed to user search agent queries,
to compile classifiers in machine learning environments, etc.