Oh btw, I found another strange behavior while chasing previous "bug":on a clean and new workspace I can't get PDT to save some of my new preferences !For example if I choose a new formatter under Preferences -> PHP -> Code Style -> Formatteror when I check "Remove trailing whitespace" and "Format source code" under Preferences -> PHP -> Editor -> Save Actionsthese options will work so long eclipse is started, but once I restart eclipse, these options will be undone (ie not saved) !

Thierry, your help is invaluable :) You pointed me into the right direction. I didn't check the scenario with dependent projects but it looks like it is the root cause of the problems.

Thanks again for your help and have a nice holiday leave! :)

On 5/4/2016 5:53 PM, thierry blind wrote:

Maybe in my case I simply though it does loop forever, because I had a looooooot of big projects depending on same big project "A" (by doing Properties -> PHP -> Include Path -> Projects -> Add -> projet A).
And it took forever with a big amount of place taken by org.eclipse.dltk.core.index.lucene
Strangely it was much faster on a new and clean workspace :S
If I have for example 5 projects that depends on a common project "A", the project "A" will then be indexed 5 times.
In my case with H2 indexer it took 1.1 GB, with the Lucene indexer it takes more than 3GB on a safe and new workspace!
So really sorry for pointing in the wrong direction :/
And yes there is still room for improvements ;)

OK, great. For now I have found something else while using your workspace that is not OK. Even if indexing finishes OK there are some source modules being re-indexed while closing/restarting IDE. At least it might give me some clues what might happened as I
think that the problem is related somehow to storing/reading source files timestamps as this is the place where some redundant indexing might be triggered.

On 5/4/2016 5:07 PM, thierry blind wrote:

I'll repackage another workspace and tell you when it's available ok ? Maybe I removed too much...

I used your workspace in my dev env but unfortunately indexer finished without a problem. It only took some time to close the PDT running threads (after closing IDE) as Lucene was merging and closing huge indexes (for about 45-60 seconds) but it finished it
gracefully. Anyway I will try to do my best to reproduce it ;)

On 5/4/2016 4:13 PM, thierry blind wrote:

With pleasure ;)
You can send it at my email address: thierryblind@xxxxxxx
Kaloyan, if you're also interested, tell me ok?

I really hope you will also reproduce this bug on your eclipse, I will go crazy if I'm the only one to have it! lol
Of course each time you start your eclipse you have to delete .metadata/org.eclipse.dltk.core.index.lucene

And some additional infos on what's installed in my eclipse Mars.2 Release:

That would be great! Thank you one more time for all the feedback! This zipped workspace would be very appreciated, I will send you soon my private address that you can send the google drive link at.

On 5/4/2016 3:50 PM, thierry blind wrote:

It won't be possible at work for Skype sorry :S
And I followed progress bar since first try ;)
Here is what happen : I have PHP projects A, B, C and D. Projects B, C and D depend on project A.
So indexer starts with project A, continues with project B (only few files), goes back to A, does a little bit from C, goes back to A, does a little bit from D, and then
it seems it only loops inside of A. It seems that if I only have project A open in my workspace, indexer works correctly and then I can open project B and indexer still works correctly, and so on...

I'll try to exclude most of the projects of my workspace , zip my workspace and share it on my google drive, would it be ok?
The workspace will look awful with lot of broken things, but mainly it will continue to loop forever (hopefully also for you!), it's the most important no ? ;)
Of course there will still be private stuff inside, please give me a private email address, so I can send you the google drive link ;)

Unfortunately nothing special in the thread dumps. Could you possibly take a look at the indexing progress bar in "Progress" view while having this infinite indexing? Does the names of the files being indexed change or progress stuck only on a one particular
file? Maybe it would be possible for you to arrange some short Skype call so you can show me what is going on?

First of all thank you Thierry for spending some time on testing new indexer. The problem that you found doesn't look good as you already said. I will try to find the root cause as fast as possible. Could you possibly attach your entire .log file?

Greetings,
Bart

On 5/3/2016 7:23 PM, thierry blind wrote:

Hi Kaloyan,
I've restarted everything (no more errors for now), first time it was probably my fault (I had an eclipse zombie process when I started a new eclipse instance).
I'm working on an existing workspace for now.
But there is still something very very wrong here.
I have very big projects (mainly because I use ORMs like Propel to generate a lot of PHP classes), and it seems that the indexer loops over same (Propel) files again and again.
The size of the "org.eclipse.dltk.core.index.lucene" directory is much bigger than the "org.eclipse.dltk.core.index.sql.h2 directory" !!!!
And indexing is still not ended... I will probably stop it before it fills my hard drive.
I will continue tests tomorrow, sadly tomorrow will be the last day I can do some tests before next 2-3 weeks :/

Hi Kaloyan,
I just updated to latest PDT snapshot 4.0.0.201605031251 (from http://download.eclipse.org/tools/pdt/updates/latest-nightly),
but I can't get the Lucene indexer to work.
Before filling a bug report I wanted to know what I do wrong:
1) I deleted .metadata\.plugins\org.eclipse.dltk.core.index.sql.h2 from my workspace
2) I started eclipse, now I get an infinite loop of following errors:

org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@C:\regroup\workspace10\.metadata\.plugins\org.eclipse.dltk.core.index.lucene\index\bc981875-9762-4936-b1d8-fbd9b4d8d2a3\timestamps\write.lock
at org.apache.lucene.store.Lock.obtain(Lock.java:89)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:780)
at org.eclipse.dltk.internal.core.index.lucene.IndexContainer.getTimestampsWriter(IndexContainer.java:128)
at org.eclipse.dltk.internal.core.index.lucene.IndexContainer.delete(IndexContainer.java:193)
at org.eclipse.dltk.internal.core.index.lucene.LuceneManager.delete(LuceneManager.java:225)
at org.eclipse.dltk.internal.core.index.lucene.LuceneIndexer.indexDocument(LuceneIndexer.java:161)
at org.eclipse.dltk.internal.core.index2.SourceModulesRequest.run(SourceModulesRequest.java:81)
at org.eclipse.dltk.core.search.indexing.AbstractJob.execute(AbstractJob.java:80)
at org.eclipse.dltk.internal.core.search.processing.JobManager.run(JobManager.java:473)
at java.lang.Thread.run(Thread.java:745)

java.lang.NullPointerException
at org.eclipse.dltk.internal.core.index.lucene.IndexContainer.delete(IndexContainer.java:193)
at org.eclipse.dltk.internal.core.index.lucene.LuceneManager.delete(LuceneManager.java:225)
at org.eclipse.dltk.internal.core.index.lucene.LuceneIndexer.indexDocument(LuceneIndexer.java:161)
at org.eclipse.dltk.internal.core.index2.SourceModulesRequest.run(SourceModulesRequest.java:81)
at org.eclipse.dltk.core.search.indexing.AbstractJob.execute(AbstractJob.java:80)
at org.eclipse.dltk.internal.core.search.processing.JobManager.run(JobManager.java:473)
at java.lang.Thread.run(Thread.java:745)