After changing my grammar - which included using the options backtrack
and memoize - my content assist throws an IllegalStateException "Stack
of grammar elements seems to be corrupt.". Here is the full stacktrace:

java.lang.RuntimeException: java.lang.IllegalStateException: Stack of
grammar elements seems to be corrupt.
at
org.eclipse.xtext.ui.editor.contentassist.antlr.AbstractCont entAssistParser.getFollowElements(AbstractContentAssistParse r.java:163)
at
org.eclipse.xtext.ui.editor.contentassist.antlr.AbstractCont entAssistParser.getFollowElements(AbstractContentAssistParse r.java:84)
at
org.eclipse.xtext.ui.editor.contentassist.antlr.ParserBasedC ontentAssistContextFactory.computeFollowElements(ParserBased ContentAssistContextFactory.java:517)
at
org.eclipse.xtext.ui.editor.contentassist.antlr.ParserBasedC ontentAssistContextFactory.computeFollowElements(ParserBased ContentAssistContextFactory.java:483)
at
org.eclipse.xtext.ui.editor.contentassist.antlr.ParserBasedC ontentAssistContextFactory.computeFollowElements(ParserBased ContentAssistContextFactory.java:478)
at
org.eclipse.xtext.ui.editor.contentassist.antlr.ParserBasedC ontentAssistContextFactory.createContexts(ParserBasedContent AssistContextFactory.java:191)
at
org.eclipse.xtext.ui.editor.contentassist.antlr.ParserBasedC ontentAssistContextFactory.create(ParserBasedContentAssistCo ntextFactory.java:130)
at
org.eclipse.xtext.ui.editor.contentassist.CompletionProposal Computer.exec(CompletionProposalComputer.java:48)
at
org.eclipse.xtext.ui.editor.contentassist.CompletionProposal Computer.exec(CompletionProposalComputer.java:1)
at
org.eclipse.xtext.util.concurrent.IStateAccess$AbstractImpl. readOnly(IStateAccess.java:40)
at
org.eclipse.xtext.ui.editor.model.XtextDocument.readOnly(Xte xtDocument.java:70)
at
org.eclipse.xtext.ui.editor.contentassist.XtextContentAssist Processor.computeCompletionProposals(XtextContentAssistProce ssor.java:79)
at
org.eclipse.jface.text.contentassist.ContentAssistant.comput eCompletionProposals(ContentAssistant.java:1834)
at
org.eclipse.jface.text.contentassist.CompletionProposalPopup .computeProposals(CompletionProposalPopup.java:556)
at
org.eclipse.jface.text.contentassist.CompletionProposalPopup .access$16(CompletionProposalPopup.java:553)
at
org.eclipse.jface.text.contentassist.CompletionProposalPopup $2.run(CompletionProposalPopup.java:488)
at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator .java:70)
at
org.eclipse.jface.text.contentassist.CompletionProposalPopup .showProposals(CompletionProposalPopup.java:482)
at
org.eclipse.jface.text.contentassist.ContentAssistant.showPo ssibleCompletions(ContentAssistant.java:1660)
at
org.eclipse.jface.text.source.SourceViewer.doOperation(Sourc eViewer.java:913)
at
org.eclipse.jface.text.source.projection.ProjectionViewer.do Operation(ProjectionViewer.java:1534)
at
org.eclipse.ui.texteditor.ContentAssistAction$1.run(ContentA ssistAction.java:82)
at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator .java:70)
at
org.eclipse.ui.texteditor.ContentAssistAction.run(ContentAss istAction.java:80)
at org.eclipse.jface.action.Action.runWithEvent(Action.java:498 )
at org.eclipse.ui.commands.ActionHandler.execute(ActionHandler. java:185)
at
org.eclipse.ui.internal.handlers.LegacyHandlerWrapper.execut e(LegacyHandlerWrapper.java:109)
at org.eclipse.core.commands.Command.executeWithChecks(Command. java:476)
at
org.eclipse.core.commands.ParameterizedCommand.executeWithCh ecks(ParameterizedCommand.java:508)
at
org.eclipse.ui.internal.handlers.HandlerService.executeComma nd(HandlerService.java:169)
at
org.eclipse.ui.internal.keys.WorkbenchKeyboard.executeComman d(WorkbenchKeyboard.java:468)
at
org.eclipse.ui.internal.keys.WorkbenchKeyboard.press(Workben chKeyboard.java:786)
at
org.eclipse.ui.internal.keys.WorkbenchKeyboard.processKeyEve nt(WorkbenchKeyboard.java:885)
at
org.eclipse.ui.internal.keys.WorkbenchKeyboard.filterKeySequ enceBindings(WorkbenchKeyboard.java:567)
at
org.eclipse.ui.internal.keys.WorkbenchKeyboard.access$3(Work benchKeyboard.java:508)
at
org.eclipse.ui.internal.keys.WorkbenchKeyboard$KeyDownFilter .handleEvent(WorkbenchKeyboard.java:123)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java :84)
at org.eclipse.swt.widgets.Display.filterEvent(Display.java:152 5)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1257)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1282)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1267)
at org.eclipse.swt.widgets.Widget.sendKeyEvent(Widget.java:1294 )
at org.eclipse.swt.widgets.Widget.gtk_key_press_event(Widget.ja va:730)
at org.eclipse.swt.widgets.Control.gtk_key_press_event(Control. java:2839)
at
org.eclipse.swt.widgets.Composite.gtk_key_press_event(Compos ite.java:734)
at org.eclipse.swt.widgets.Widget.windowProc(Widget.java:1743)
at org.eclipse.swt.widgets.Control.windowProc(Control.java:4794 )
at org.eclipse.swt.widgets.Display.windowProc(Display.java:4372 )
at org.eclipse.swt.internal.gtk.OS._gtk_main_do_event(Native Method)
at org.eclipse.swt.internal.gtk.OS.gtk_main_do_event(OS.java:81 63)
at org.eclipse.swt.widgets.Display.eventProc(Display.java:1239)
at org.eclipse.swt.internal.gtk.OS._g_main_context_iteration(Na tive Method)
at org.eclipse.swt.internal.gtk.OS.g_main_context_iteration(OS. java:2224)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java :3169)
at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.jav a:2629)
at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2593)
at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:24 27)
at org.eclipse.ui.internal.Workbench$7.run(Workbench.java:670)
at
org.eclipse.core.databinding.observable.Realm.runWithDefault (Realm.java:332)
at
org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Work bench.java:663)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.j ava:149)
at
org.eclipse.ui.internal.ide.application.IDEApplication.start (IDEApplication.java:115)
at
org.eclipse.equinox.internal.app.EclipseAppHandle.run(Eclips eAppHandle.java:196)
at
org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher .runApplication(EclipseAppLauncher.java:110)
at
org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher .start(EclipseAppLauncher.java:79)
at
org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseS tarter.java:369)
at
org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseS tarter.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce ssorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe thodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java: 619)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:574)
at org.eclipse.equinox.launcher.Main.run(Main.java:1407)
at org.eclipse.equinox.launcher.Main.main(Main.java:1383)

Looking at the source code didn't give me a hint what might have gone
wrong. And searching bugzilla and google didn't help either. The only
unusual thing I do in my project is to use a post process to move some
features from subclasses to their common super class. Can this cause
this problem? Or is it caused by my grammar?

Best regards,
Matthias]]>2010-11-27T17:10:20-00:00Re: [Xtext] IllegalStateException &quot;Stack of grammar elements seems to be corrupt.&quot;https://www.eclipse.org/forums/index.php/mv/msg/200993/641778/#msg_641778

Eclipse User wrote on Sat, 27 November 2010 18:10

Originally posted by: koester.matthias.gmx.de
The only
unusual thing I do in my project is to use a post process to move some
features from subclasses to their common super class. Can this cause
this problem? Or is it caused by my grammar?

This is suspicious: Xtext performs a normalization of the Ecore before the postprocessing, so normally there can't be any need to do this yourself. It could very well be that you modify the Ecore model in a way which is incompatible with what Xtext expects (see "Ecore Model Inference" in the Xtext User Guide).]]>Meinte Boersma2010-11-27T22:08:00-00:00Re: [Xtext] IllegalStateException &quot;Stack of grammar elements seems to be corrupt.&quot;https://www.eclipse.org/forums/index.php/mv/msg/200993/641803/#msg_641803
the exception signals a violation of an invariant in the content assist
parser. It is unlikely that your Ecore model post-processing has
something to do with it. Could you please file a ticket with a
reproducable sample attached? Does it help if you turn memoization off?

I've read the documentation about the model interference and currently I
don't see another way than a post processor to shape my metamodel as I
want it to look like;-) But since my experience with Xtext is still
growing, perhaps I can get rid of the post processor in the future;-)

But my guess is that as long as Xtext works with a subclass it shouldn't
matter if a feature is defined in the superclass or the subclass!
Otherwise I would have been very surprised and it would break the
principle of least astonishement...

As Sebastian stated it seems to be either a problem with my grammar or
perhaps a bug. We will investigate it further and report back.

Best regards,
Matthias

Meinte Boersma schrieb:
> Eclipse User wrote on Sat, 27 November 2010 18:10
>> Originally posted by: koester.matthias.gmx.de
>> The only unusual thing I do in my project is to use a post process to
>> move some features from subclasses to their common super class. Can
>> this cause this problem? Or is it caused by my grammar?
>
> This is suspicious: Xtext performs a normalization of the Ecore before
> the postprocessing, so normally there can't be any need to do this
> yourself. It could very well be that you modify the Ecore model in a way
> which is incompatible with what Xtext expects (see "Ecore Model
> Inference" in the Xtext User Guide).]]>2010-11-28T10:00:31-00:00Re: [Xtext] IllegalStateException &quot;Stack of grammar elements seems to be corrupt.&quot;https://www.eclipse.org/forums/index.php/mv/msg/200993/641809/#msg_641809
Originally posted by: koester.matthias.gmx.de

Hi Sebastian,

Removing memoize=true didn't help, but it looks like my scope providers
are causing the problem: I copied my grammar to a new xtext project
without my scope providers and there the content assist worked. Of
course without the interesting stuff ;-)

I will investigate this problem further, but just wanted to hear if you
think that the problem can also be caused by my scope providers?

Best regards,
Matthias

Sebastian Zarnekow schrieb:
> Hi Matthias,
>
> the exception signals a violation of an invariant in the content assist
> parser. It is unlikely that your Ecore model post-processing has
> something to do with it. Could you please file a ticket with a
> reproducable sample attached? Does it help if you turn memoization off?
>
> Regards,
> Sebastian]]>2010-11-28T11:03:01-00:00Re: [Xtext] IllegalStateException &quot;Stack of grammar elements seems to be corrupt.&quot;https://www.eclipse.org/forums/index.php/mv/msg/200993/641811/#msg_641811
at a first glance I'd doubt that a scope provider implementation can
cause this exception.

Am 28.11.10 12:03, schrieb Matthias Köster:
> Hi Sebastian,
>
> Removing memoize=true didn't help, but it looks like my scope providers
> are causing the problem: I copied my grammar to a new xtext project
> without my scope providers and there the content assist worked. Of
> course without the interesting stuff ;-)
>
> I will investigate this problem further, but just wanted to hear if you
> think that the problem can also be caused by my scope providers?
>
> Best regards,
> Matthias
>
>
> Sebastian Zarnekow schrieb:
>> Hi Matthias,
>>
>> the exception signals a violation of an invariant in the content
>> assist parser. It is unlikely that your Ecore model post-processing
>> has something to do with it. Could you please file a ticket with a
>> reproducable sample attached? Does it help if you turn memoization off?
>>
>> Regards,
>> Sebastian]]>Sebastian Zarnekow2010-11-28T11:09:56-00:00Re: [Xtext] IllegalStateException &quot;Stack of grammar elements seems to be corrupt.&quot;https://www.eclipse.org/forums/index.php/mv/msg/200993/641815/#msg_641815
Originally posted by: koester.matthias.gmx.de

I encountered another strange thing: When I disable the backtrack and
memoize option for my parser.antlr.XtextAntlrUiGeneratorFragment the
generation fails. That's fine, but when I now regenerate everything with
option backtrack=true I get the following ANTLR error:

error(10): internal error:
org.antlr.tool.Grammar.createLookaheadDFA(Grammar.java:864): could not
even do k=1 for decision 10

Now the strange thing:
When I generate everything a second time, the ANTLR error doesn't show
up! As far as I can see my MWE should clean the generated source before
each run, but now it seems to store some state between runs... Or why
does my generator fail during the first run and fails on the second run?

Regards,
Matthias

Sebastian Zarnekow schrieb:
> Hi Matthias,
>
> at a first glance I'd doubt that a scope provider implementation can
> cause this exception.
>
> Regards,
> Sebastian]]>2010-11-28T11:36:13-00:00Re: [Xtext] IllegalStateException &quot;Stack of grammar elements seems to be corrupt.&quot;https://www.eclipse.org/forums/index.php/mv/msg/200993/641823/#msg_641823
Antlr uses an internal timeout when it's analyzing the grammar. Your
observation should not have something to do with first and second time
generation.
See here: http://20000frames.blogspot.com/2010/09/dealing-with-could-n ot-even-do-k1-for.html

Am 28.11.10 12:36, schrieb Matthias Köster:
> I encountered another strange thing: When I disable the backtrack and
> memoize option for my parser.antlr.XtextAntlrUiGeneratorFragment the
> generation fails. That's fine, but when I now regenerate everything with
> option backtrack=true I get the following ANTLR error:
>
> error(10): internal error:
> org.antlr.tool.Grammar.createLookaheadDFA(Grammar.java:864): could not
> even do k=1 for decision 10
>
> Now the strange thing:
> When I generate everything a second time, the ANTLR error doesn't show
> up! As far as I can see my MWE should clean the generated source before
> each run, but now it seems to store some state between runs... Or why
> does my generator fail during the first run and fails on the second run?
>
> Regards,
> Matthias
>
>
> Sebastian Zarnekow schrieb:
>> Hi Matthias,
>>
>> at a first glance I'd doubt that a scope provider implementation can
>> cause this exception.
>>
>> Regards,
>> Sebastian]]>Sebastian Zarnekow2010-11-28T14:31:04-00:00Re: [Xtext] IllegalStateException &quot;Stack of grammar elements seems to be corrupt.&quot;https://www.eclipse.org/forums/index.php/mv/msg/200993/641895/#msg_641895
> I've read the documentation about the model
> interference and currently I don't see another
> way than a post processor to shape my
> metamodel as I want it to look like;-)

have you tried instantiating an existing meta model? You need not use the one generated automatically.

I fixed my problem by patching the
org.eclipse.xtext.ui.editor.contentassist.antlr.AbstractCont entAssistParser
class. Stripping down my xtext project to a reasonable size requires
more time than I can invest right now, but my example is open sourced at
google code. The patch is actually very easy, to me it looks like an if
statement in method getFollowElements(final
AbstractInternalContentAssistParser parser, AbstractElement
elementToParse, String[] ruleNames, int startIndex)
isn't correct.

So how can we proceed to solve the problem? Should I file a bug request
with a pointer to my open source project or is it enough when I add the
patch to my bug request?

Best regards,
Matthias

Sebastian Zarnekow schrieb:
> Hi Matthias,
>
> Antlr uses an internal timeout when it's analyzing the grammar. Your
> observation should not have something to do with first and second time
> generation.
> See here:
> http://20000frames.blogspot.com/2010/09/dealing-with-could-n ot-even-do-k1-for.html
>
>
> Regards,
> Sebastian]]>2010-11-29T08:05:47-00:00Re: [Xtext] IllegalStateException &quot;Stack of grammar elements seems to be corrupt.&quot;https://www.eclipse.org/forums/index.php/mv/msg/200993/641915/#msg_641915
Originally posted by: koester.matthias.gmx.de

Hi Alex,

Thx for your hint, but I already knew that I can start from a metamodel
and then create a Xtext grammar to map to it ;-)

Since my post processor doesn't seem to cause the problem, I'm fine with
my current approach ;-)
I like the DRY principle and it's great that Xtext is supporting it. For
me one of the main benefits of using Xtext is the fast turn around time.
Before using Xtext I used ANTLR and DLTK. That gave me the full power of
ANTLR (like predicates, etc.), but I had to synchronize my grammar with
my metamodel manually. That was really time consuming and wasn't fun at
all.
Another stumbling block was using DLTK. It's really a great framework,
but it's still lacking documentation. And the source code isn't very
easy to understand, especially when compared with Xtext! And the Xtext
community is much bigger and much nicer;-)

Best regards,
Matthias

Alexander Nittka schrieb:
> Hi,
>
>> I've read the documentation about the model interference and currently
>> I don't see another
>> way than a post processor to shape my metamodel as I want it to look
>> like;-)
>
> have you tried instantiating an existing meta model? You need not use
> the one generated automatically.
>
> Alex]]>2010-11-29T08:37:02-00:00Re: [Xtext] IllegalStateException &quot;Stack of grammar elements seems to be corrupt.&quot;https://www.eclipse.org/forums/index.php/mv/msg/200993/641923/#msg_641923
a patch and a test case would be great.

Am 29.11.10 09:05, schrieb Matthias Köster:
> Hi Sebastian,
>
> I fixed my problem by patching the
> org.eclipse.xtext.ui.editor.contentassist.antlr.AbstractCont entAssistParser
> class. Stripping down my xtext project to a reasonable size requires
> more time than I can invest right now, but my example is open sourced at
> google code. The patch is actually very easy, to me it looks like an if
> statement in method getFollowElements(final
> AbstractInternalContentAssistParser parser, AbstractElement
> elementToParse, String[] ruleNames, int startIndex)
> isn't correct.
>
> So how can we proceed to solve the problem? Should I file a bug request
> with a pointer to my open source project or is it enough when I add the
> patch to my bug request?
>
> Best regards,
> Matthias
>
> Sebastian Zarnekow schrieb:
>> Hi Matthias,
>>
>> Antlr uses an internal timeout when it's analyzing the grammar. Your
>> observation should not have something to do with first and second time
>> generation.
>> See here:
>> http://20000frames.blogspot.com/2010/09/dealing-with-could-n ot-even-do-k1-for.html
>>
>>
>> Regards,
>> Sebastian]]>Sebastian Zarnekow2010-11-29T09:31:02-00:00Re: [Xtext] IllegalStateException &quot;Stack of grammar elements seems to be corrupt.&quot;https://www.eclipse.org/forums/index.php/mv/msg/200993/995907/#msg_995907
I think I have the same problem and I want to know if this patch is available ?

Thank you !

Regards,

Sylvain]]>Sylvain Heraud2012-12-31T13:29:40-00:00Re: [Xtext] IllegalStateException &amp;quot;Stack of grammar elements seems to be corrupt.&amp;quot;https://www.eclipse.org/forums/index.php/mv/msg/200993/996484/#msg_996484
> Hi Matthias, Hi Sebastian.
>
> I think I have the same problem and I want to know if this patch is
> available ?
>
> Thank you !
>
> Regards,
>
> Sylvain