Stephen Leake <stephen_leake@...> writes:
> Next question: how do I compile a wisent grammar file in Emacs 24.2?
>
> I'd like to build a very simple grammar for a small subset of Ada, so I
> can play with it interactively and see if I can do what I need for an
> indentation engine.
>
> In Emacs 23, there was a file '../etc/grammars/wisent-grammar.el' that
> would read a .wy file and produce a -wy.el file. I don't see the
> equivalent in Emacs 24.2.
>
> Do I need to download something from sourceforge?
So apparently the answer to that is "yes"; in cedet-1.1/semantic/wisent
there is a Makefile that compiles .wy files. Copied bits of that to my
Makefile, and now I have a file wisent-ada-simple-wy.el.
Which defines a function wisent-ada-simple-wy--install-parser, but I'm
having trouble getting that to run.
I think it may be a compatibility issue between cedet-1.1 and Emacs 24;
I'll try cedet-1.0.1 (tomorrow :)
--
-- Stephe

Feedback inserted below:
> Date: Fri, 5 Oct 2012 10:10:38 +0200
> From: eliz@...
> Subject: Re: [cedet-semantic] Latest CEDET on BZR does not compile with emacs 24.1
> To: vincent.b.1@...
> CC: cedet-semantic@...; deng@...; emacs-devel@...
>
> > From: Vincent Belaïche <vincent.b.1@...>
> > CC: "cedet-semantic@..."
> > <cedet-semantic@...>, emacs-devel <emacs-devel@...>
> > Date: Fri, 5 Oct 2012 07:18:53 +0200
> >
> > Thank you Eli for your kind answer. I tried it with the mingw32-make a link to which you have sent to me. It is correct that it does not use MSYS paths like /c/Programme/GNU/installation/cedet-install/cedet/lisp/cedet/loaddefs.el, but the same kind of path that pwd -W outputs, i.e. c:/Programme/GNU/installation/cedet-install/cedet/lisp/cedet/loaddefs.el for the same file. EMACS is supposed to understand that kind of path. However that still does not work:
>
> My crystal ball says that pwd is an MSYS program. IOW, you still have
> MSYS's /bin directory on your general PATH,
Your crystal ball is perfectly correct
> and those MSYS tools,
> especially Bash, are being invoked by the MinGW Make.
>
> If you are invoking Make from the MSYS Bash console window, don't;
> invoke it from a regular cmd window instead.
I tried both actually, but, true, MSYS's /bin was in the PATH in both cases
> If you added MSYS's /bin
> directory to your PATH such that even the cmd window gets that PATH,
> edit your environment variables and remove the MSYS /bin directory
> from the PATH. Then these problems will go away, and you will be
> faced only with real problems: the Unixy features used by the Makefile
> that don't work on Windows. These you will have to solve, but at
> least you will be fighting known issues.
Well, call this the "real problems" if you like, I am slightly sceptical on what the real problem really is, most Makefiles I have seen in my life are delegating some file manipulation tasks to some bash shell. Even the documentation of Make is most of the time doing this assumption --- e.g. see those `rm ...' in the documentation of `clean' targets.
The reason why so is that GNU make may have lots of thing in its sleeve, but, AFAIK, it does not have builtins to do basical file manipation functions. So it has to resort to some shell call, most of the time Bash.
In contrast to GNU make, ant has the filterset, delete, mkdir, and copy tasks. So one does not need to delegate those basic things to any shell, and build.xml files are made more portable this way.
Considering CEDET I tried what you said, I wrote an MSDOS batch file with this content (the first line ensures that I have both mingw32-make and emacs in the PATH, but no more :
------- begin -------SET PATH=c:\msys\1.0\mingw\bin;c:\Programme\GNU\emacs-24.1\bin
SET RM=DEL
CD cedet
mingw32-make
CD ..
------- end -------
and that failed quite soon with this output:
------- begin -------
C:\Programme\GNU\installation\cedet-install>SET PATH=c:\msys\1.0\mingw\bin;c:\Programme\GNU\emacs-24.1\bin
C:\Programme\GNU\installation\cedet-install>SET RM=DEL
C:\Programme\GNU\installation\cedet-install>CD cedet
C:\Programme\GNU\installation\cedet-install\cedet>mingw32-make
Emacs version: 24.1.1 nil on windows-nt
Removing loaddefs.el files from subprojects.
-f était inattendu.
mingw32-make: *** [clean-autoloads] Error 255
C:\Programme\GNU\installation\cedet-install\cedet>CD ..
------- end ------- As far as I can understand this failure happened when this statement was met:
@$(foreach proj,$(PROJECTS_AUTOLOADS),cd $(CURDIR)/$(proj) && if [ -f $(LOADDEFS) ];then $(RM) -f $(LOADDEFS);fi;)
Quite surprising DOS does not bark on `cd $(CURDIR)/$(proj)' despite the `/' inside it, but the `if [ -f ... ]; ....; fi' instead of `IF EXIST ... ( ... )' was a bit too much asking of DOS / Bash similarity.
It just seems to me quite challenging to write Makefile in a portable way while --- sorry for repeating myself --- GNU Make is missing those tasks called filterset, delete, mkdir, and copy in ant.GNU Make simply does not have these things built-in, this is why most Makefile delegate them to Bash, and one can see find, rm, mkdir and cp bash commands instead, which means that you get into troubles when you want to use this Makefile on a plaform without Bash.
Here we are talking on doing macro definition to have some portable tasks like filterset, delete, mkdir, and copy in ant --- i.e. not the same macro definition in DOS and in a unixy shell, but that reasoning is quite unstatisfactory, because that means that every time you are considering a new shell you will need to create again such a set of macros --- or better, user defined function that one would $(call ...).
So, you may argue that the real problem is that Makefile call some bash commands, but I am more opinionated that the real problem is that GNU Make is missing the builtin features that would allow one to do without those bash command in the statements in a standard way, and without user defined macros/function to ensure portability.
>
> > Debugger entered--Lisp error: (file-error "Opening output file" "no such file or directory" "d:/devel/emacs/release/emacs24/emacs-24.1/lisp/c;c:msys.0ProgrammeGNUinstallationcedet-installcedetlispcedetloaddefs.el")
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> See this nonsense? That's MSYS file syntax conversion in action.
[... SNIP ...]
>
Thank you again for your commenting on it: I must admit that I was not aware that MSYS was so invasive in preprocessing arguments like that, I could not even imagine it, I had thought that emacs was the one doing those kind of argument preprocessing.
That is all the more surprising that the string that is transformed is within some simple-quotes.
> > What happens is that path `c:/Programme/GNU/installation/cedet-install/cedet/lisp/cedet/loaddefs.el' in the command line transformed by EMACS into `c;c:\msys\1.0\Programme\GNU\installation\cedet-install\cedet\lisp\cedet\loaddefs.el'.
>
> Yes. This is what MSYS does, and it does that for a good reason. The
> problem is that the algorithms MSYS applies for this conversion
> misfire when the program being invoked has its own ideas about the
> syntax of the command-line arguments. This is one such case.
> (Invoking a natively-compiled Sed is another prominent case where the
> MSYS file-name conversion badly misfires.)>
Ok, I see, unless if you do the 's/find-this/replace-by-that/g' with some other character than `/', e.g. like that 's!find-this!replace-by-that!g'
But your comment is quite interesting: would that mean that if emacs was compiled to be an MSYS application then it would be able to prevent MSYS from doing mischiefs by telling to MSYS which argument is supposed to be a path and which is not ?
I mean that EMACS people boasting that EMACS is ported on most known platform seem not to consider MSYS as a platform on its own, i.e. it seems that EMACS Win32 port does not test at all whether MSYSTEM environment variable is set to MSYS or to MINGW32, nor whether OSTYPE is set to msys ...
>
> > Probably the reason for doing that is quite valid: the path should have been ` c:\Programme\GNU\installation\cedet-install\cedet\lisp\cedet\loaddefs.el' in the first place.
>
> No. (Btw, using "D:/foo/bar" syntax of file names is perfectly OK on
> Windows, because the Windows file I/O APIs understand forward slashes
> very well, but that's not the issue here.) The problem is elsewhere:
> this command-line argument is _not_ a file name. It is a Lisp string
> that will be interpreted as a file name by Emacs. But MSYS does not
> (and cannot) have the slightest idea about valid syntax of Lisp
> strings; in particular, it doesn't know that a backslash in a Lisp
> string needs to be doubled. So it ruins the command line by its
> simplistic conversion.>
> >My thought are rather that is it hard to write makefile that are portable because GNU Make is lacking the following features:
> >
> >
> > - a predefined variable, e.g. $(FS) expanding to the file separator, i.e. \ for MSWindows and / for Unixy shell
>
> Why do you need this? As I said, Windows programs, Emacs included,
> understand forward slashes in file names quite well. The only one
> that might have problems is cmd.exe, but you should not need to invoke
> cmd.exe's commands too much, if at all.
Well, for instance, as far as I remember DEL is an MSDOS internal command, so at least for all the clean and such like targets --- which quite common in any Makefile --- you need those backslash and not forward-slash, if you assume that you have only MSDOS to rely on, and not MSYS in your path, or at least RM variable defined to MSYS's rm.exe
> And if you do need internal
> cmd commands, there are the 'subst' and 'patsubst' Make functions you
> can use to convert forward slashes to backslashes on the cmd command
> lines.
>
But then that kind of mean using the same sort of tricks to which I resorted to compile with MSYS. I was also doing those subst to have the backslashes in the first place and be sure that no problem would happen become some SW (be it MSYS or EMACS) would do some mistake in badly converting forward slashed to backward slashes
> Also, do you know that Make converts all environment variables to Make
> variables, and expands them?
>
> > - a predefined variable, e.g. $(CFS), expanding to the file separator within a C-string-like environment, i.e. \\ for MSWindows and / for Unixy shell
>
> Again, you should not need this at all. This problem does not exist,
> as long as you invoke Emacs from the Makefile.
>
> > - a function, e.g. $(pathconvert ...) to convert a path name in the the Unixy like format that make internally uses with wildcards and $(abspath ...) returned value into the format used on the platform where make is running, i.e. in my case `c:/Programme/GNU/installation/cedet-install/cedet/lisp/cedet/loaddefs.el' would be converted to `c:\Programme\GNU\installation\cedet-install\cedet\lisp\cedet\loaddefs.el'
>
> Not needed, see above.
>
> > - and a function, e.g. $(cpathconvert ...) which would do the same thing but with output to go within some C-string-like environement, i.e. in my case
> > `c:/Programme/GNU/installation/cedet-install/cedet/lisp/cedet/loaddefs.el' would be converted to `c:\\Programme\\GNU\\installation\\cedet-install\\cedet\\lisp\\cedet\\loaddefs.el'
>
> Again, not needed.
>
> > Without those basic things, one has no other resorts than the kind of tricks which I played to make the Makefile platform independent, and which, I agree, may be fragile, though working ones.
>
> You will have to explain why you at all need these basic things.
>
> > Please note that this is why I mentioned ant: in contrast to GNU Make, ant has that sort of tricks that allow to do path name manipulations.
>
> GNU Make has enough up its sleeve for this job, and many similar ones,
> believe me.
>
I would be more than quite happy to believe you, but it seems that I don't have enough expertize in GNU make for having this belief. And, this lack of expertize --- of lack of things up the sleeve of GNU make, let us see --- is not just mine: see it, CEDET's Makefile's are not portable to DOS according to your criterion that the real problem is the use of Bash commands. E.g. CEDET Makefile's call bash specific command `if [ -f .... ]; then ... fi'. But surely CEDET people who created EDE know far more about writing Makefiles than I do. However CEDET's Makefile could be portable to Win32 if you take it for granted that GNU Make cannot leave without some Unixy shell nearby --- so MSYS under MSWindow ---, and therefore that the only remaining issue is path manipulation. Hence the FS, CFS, pathconvert and cpathconvert which I was suggesting.
VBR, Vincent.
>
>

Stephen Leake <stephen_leake@...> writes:
> I found the top level function 'bovinate'; when I run it on a python
> buffer (for example), it outputs semantic information; that looks very
> promising. However, I can't trace via edebug down into
> `semantic-parse-region'; that is defined by define-overloadable-function.
>
> I don't see where python-mode defines an overload for that, and I also
> don't see where the default is defined.
Well, I figured out this part; the default of an overloaded symbol is
<name>-default.
So I found the core loop for semantic-parse-region; I'll have to study
it some more.
Next question: how do I compile a wisent grammar file in Emacs 24.2?
I'd like to build a very simple grammar for a small subset of Ada, so I
can play with it interactively and see if I can do what I need for an
indentation engine.
In Emacs 23, there was a file '../etc/grammars/wisent-grammar.el' that
would read a .wy file and produce a -wy.el file. I don't see the
equivalent in Emacs 24.2.
Do I need to download something from sourceforge?
--
-- Stephe

I'm rewriting the indentation engine for Emacs Ada mode. The trigger was
the recent sytax introduced by Ada 2012; it breaks the old indentation
engine.
I've got a smie-based engine working fairly well, but it has a major
problem; there are some keywords that can only be properly recognized by
doing a full parse from the start of the file.
So I'm looking at semantic, which could be a better parser for that
style.
First, is anyone aware of an indentation engine based on semantic? A
working example would be a big help.
For an indentation engine, I need to cache information for each found
token; mainly where the start of the enclosing statement is.
I'm having a hard time getting started; I can't find the main parsing
loop, to see what hooks are available for caching things.
I found the top level function 'bovinate'; when I run it on a python
buffer (for example), it outputs semantic information; that looks very
promising. However, I can't trace via edebug down into
`semantic-parse-region'; that is defined by define-overloadable-function.
I don't see where python-mode defines an overload for that, and I also
don't see where the default is defined.
How do I find the local binding for an overloadable function? I tried
(mode-local-symbol 'semantic-parse-region) in the python buffer; that
returned nil.
--
-- Stephe

[... SNIP ...]
> > BTW, I still have the problem with
> >
> > Package lmenu is obsolete!
> > Package sregex is obsolete!
> > Are these packages from standard emacs ?
>
> No. But CEDET does not use these packages. They're probably loaded from
> your init- or site-file.
>
> -David
>
Thank you for the feedback, that is probably loaded by JDEE, as I already wrote to you I needed an up-to-date CEDET to use JDEE.
VBR,
Vincent.
PS: resending as I did Reply instead of Reply-all the first time.

Vincent Belaïche writes:
> Target Database... 0 compiled, 16 up to date.
Well, it seems almost everything seems to be already compiled from your
previous efforts.
> Then I did some make clean-all, and then I tried M-x cedet-build-in-this-emacs,
> and then I got this:
No, that's not working (yet). Please only do
emacs -Q -l cedet-build.el -f cedet-build
on a CEDET clean checkout. Everything else is not tested and will
probably not work.
> /c/Programme/GNU/installation/cedet-install/cedet>emacs -batch -l
> cedet-build.el -f cedet-build
Not the right command. Do not use '-batch', but use '-Q'.
> BTW, I still have the problem with
>
> Package lmenu is obsolete!
> Package sregex is obsolete!
> Are these packages from standard emacs ?
No. But CEDET does not use these packages. They're probably loaded from
your init- or site-file.
-David

Vincent Belaïche writes:
> I am not going to re-write all the CEDET Makefiles just to use an
> MSWindows make engine. I've read from David that those Makefiles are
> generated by EDE, so if ever there was something to do, it would be
> better to upgrade EDE to generates those MSWindows makefile, or
> anything working well under MSWindows without the need for a Unixy
> shell (e.g. ant build.xml files).
Yes, that would be the way to go. You can generate pretty much any kind
of build script using EDE. In fact, if you look into our Elisp build
script cedet-build.el, you'll see that it also calls EDE to build the
targets without using Make. (It calls `project-compile-target' on the
target, which is an object; those methods are defined in
ede/proj-elisp.el and in semantic/ede-grammar.el for the
byte-compilation and parser generation, resp.).
> Anyhow, CEDET can be built with some lisp only script that will be
> soon fixed, so even this thing is not needed.
I think I fixed it now. Please check out latest CEDET from bzr and try
emacs -Q -l cedet-build.el -f cedet-build
Please let me know if you see any errors.
-David

On 10/05/2012 10:48 AM, kototama kototama wrote:
>> (def () x 42)
>>
>> should be converted lexically into:
>>
>> DEF PAREN_BLOCK SYMBOL
>
> There must be a problem in the lexer definition or usage because even
> if I used the
> wisent-clojure-wy--<block>-block-analyzer
>
> I get LPAREN and RPAREN with semantic-lex-debug instead of getting PAREN_BLOCK.
Is the variable `semantic-lex-depth' set to something larger than 0? I
looked at your code, but I didn't see a setting for it.
That depth variable will cause the lexer to dig into lists instead of
creating blocks.
Alternately, if the lexer for parens shows up in front of the BLOCK
analyzer, it will do that first, and skip the blocks. Your regex
analyzer in particular looks like it might match parens. Unless you
need it right off, I'd comment that one out and see how it goes.
Your best bet is to build up code examples, and use semantic-lex-debug,
and make sure you are getting the right lexical token stream before
worrying much more about the rules in your grammar.
Eric