ODB IndexActivation NullPointerException

#1

Hi,

We have created indexes on an existing database, which has 4.3 million objects. As soon we create first query, NullPointerException is thrown:

Exception in thread "ODB IndexActivation" java.lang.NullPointerException
at com.objectdb.o.IIX.g(IIX.java:69)
at com.objectdb.o.IIX.f(IIX.java:58)
at com.objectdb.o.IXS.i(IXS.java:96)
at com.objectdb.o.IIS.p(IIS.java:55)
at com.objectdb.o.IXM.Vc(IXM.java:438)
at com.objectdb.o.PBI.C(PBI.java:146)
at com.objectdb.o.PBI.q(PBI.java:115)
at com.objectdb.o.MBI.VC(MBI.java:83)
at com.objectdb.o.MST.Vn(MST.java:783)
at com.objectdb.o.IXM.D(IXM.java:363)
at com.objectdb.o.IXM.run(IXM.java:338)
at java.lang.Thread.run(Thread.java:745)
* Batch ended at 88

I've read that: "Indexing large amount of data may take considerable processing time."

Do we have to wait until index activation is finished before we can start using the database or something else is causing the problem?

A new index is not being used until its building is completed it but meanwhile you should be able to use the database normally. The stack trace indicates an exception in the new index building rather than in executing a query, so apparently it is not related to the query that you ran but to the background index building, which is a new ObjectDB feature.

Build 2.6.7_07 includes an attempt to fix this exception. Please try it.

The messages are related to rebuilding an index, e.g. when a new index is defined for an existing database. Indexes are built in the background starting version 2.6.6, and apparently the last related code change was in build 2.6.7_07.

It is unclear why the application stops working and how a NullPointerException is thrown.

If you use embedded mode please check if the connection time to the database affects. If the database is closed before building the index is completed then the work is discarded and another attempt is done next time. So maybe opening and closing the database for very short periods (in which index rebuilding cannot be completed) may be related.

The index building is done in a separate thread that starts when the database is opened and runs until completion. The name of the thread in build 2.6.9_08 is "ODB-IndexActivation" (it was "ODB IndexActivation" until version 2.6.9_07). You may be able to locate the thread in embedded mode and delay closing the database until its completion (e.g. by using join).

We had the same issue using the version 2.6.1 and waited for a new version to fix. Unfortunatly the new version 2.7-1_01 does not fix that problem as it should do, according to the release notes. We could repeatable cause this error, when opening an old database (created from version 2.6.1) with the new library, do some short thing and close it again.

Using the workaround from user 'btc_es' avoids the exception. After the workaround has taken place, we could run the testcase (on the newly indexed database) several times without the workaround and without any exceptions.

Now I'm a little bit unshure, if we should rely on this workaround and wether we need it every time after opening the database or only once a time after changing the indexes.

In our scenario (exspecially update) it may happen that the database ist opened only for a short time. What if the indexer will start his work in this moment.

edit1: I Just recognized, that the behavior of 2.7.1_01 is not completly the same as in 2.6.7, escpecially it's not a null pointer exception. Instead it seems, that closing the database causes the ODB-IndexActivation Thread to complain about the now unreadable database.

edit2: I played around a little bit with our testcase and repeated the task several times before calling the workaround. It seems that once the workaround was called, every thing went fine and there was no damage at the database. The old library (I think it was 2.6.7) has sometimes damaged the whole database, so we did not dare to deliver it to our customers.

Exception in thread "ODB-IndexActivation" com.objectdb.o.UserException: Failed to read from file 'U:\Arne_Temp\TestDB_Arne_objectd_jar\dbtest\AdvoluxData\db\anwalt.objectdb'
at com.objectdb.o.MSG.d(MSG.java:75)
at com.objectdb.o.LFL.UA(LFL.java:1157)
at com.objectdb.o.LFL.UA(LFL.java:1126)
at com.objectdb.o.PAG.au(PAG.java:927)
at com.objectdb.o.PGC.C(PGC.java:283)
at com.objectdb.o.SNP.G(SNP.java:436)
at com.objectdb.o.OBI.v(OBI.java:448)
at com.objectdb.o.OBI.s(OBI.java:357)
at com.objectdb.o.OBI.r(OBI.java:274)
at com.objectdb.o.MBI.c(MBI.java:51)
at com.objectdb.o.MBI.VC(MBI.java:71)
at com.objectdb.o.MST.U8(MST.java:783)
at com.objectdb.o.IXM.E(IXM.java:379)
at com.objectdb.o.IXM.run(IXM.java:351)
at java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException: Stream Closed
at java.io.RandomAccessFile.seek0(Native Method)
at java.io.RandomAccessFile.seek(Unknown Source)
at com.objectdb.o.LFL.UA(LFL.java:1153)
... 13 more

Of course this may happen, if we close the database, while the indexer is working. Is there any legal way to detect, wether objectdb is really finished with all activities that may be proceed and should not be stopped before closing the database.