Hi,
I thought that I could play around a little bit with DMD parser. I have
DMD 1.030 parser sources, and I made a simple makefile for just compiling
those sources (with gcc 4.3.1).
---
Some questions:
- These are C++ sources, not C sources, although the suffix is .c and .h,
right? A little bit sad, I thought that I could wrap the parser to D :( I
have not much interest playing with C++ at freetime...
- Are these files meant to be compiled alone without making modifications
to files? For example:
- In "attrib.c", if linux is defined, it tries to include mem.h from "../
root"; why not the one in the src/ directory?
- There is no such file that "id.h", which probably contains class/struct/
enum "Id"
- init.h does not include "hdrgen.h", while uses HdrGenState
---
How these parser codes are meant to be used? Am I missing something?

Hi,
I thought that I could play around a little bit with DMD parser. I have
DMD 1.030 parser sources, and I made a simple makefile for just compiling
those sources (with gcc 4.3.1).
---
Some questions:
- These are C++ sources, not C sources, although the suffix is .c and .h,
right? A little bit sad, I thought that I could wrap the parser to D :( I
have not much interest playing with C++ at freetime...
- Are these files meant to be compiled alone without making modifications
to files? For example:
- In "attrib.c", if linux is defined, it tries to include mem.h from "../
root"; why not the one in the src/ directory?
- There is no such file that "id.h", which probably contains class/struct/
enum "Id"
- init.h does not include "hdrgen.h", while uses HdrGenState
---
How these parser codes are meant to be used? Am I missing something?

They aren't. They're just meant to be for anyone who is interested in
taking them and trying to build a compiler or other tool out of them.
They don't compile into anything useful as-is.
I think there's a project, DMDFE, based on those sources that is
compileable though. http://www.dsource.org/projects/dmdfe
--bb

They aren't. They're just meant to be for anyone who is interested in
taking them and trying to build a compiler or other tool out of them.
They don't compile into anything useful as-is.

Well, I thought that they could be compiled to object files, and you
would be able to wrap & link them to your own project... But if they are
meant just to be rewritten, that's of course different story.

Yes, that worked, thanks!
Next I'll try to figure out, how to easily import the parsed results to
D... Hmmh, I might have an idea; if I modify that DMDFE to output the
parse tree in some kind of ASCII stream, I might be able to construct it
in D; hmmh, that sounds somewhat silly... Well, I try to figure out
something. Again, thanks for that link! :D

Yes, that worked, thanks!
Next I'll try to figure out, how to easily import the parsed results to
D... Hmmh, I might have an idea; if I modify that DMDFE to output the
parse tree in some kind of ASCII stream, I might be able to construct it
in D; hmmh, that sounds somewhat silly... Well, I try to figure out
something. Again, thanks for that link! :D

No need to go ascii. Just wrap up the data in some structs and add some
extern(C) functions that you can call from D which return those structs.
Or if you're doing D2.0, then there's that extern(C++) thing that may work.
--bb

Yes, that worked, thanks!
Next I'll try to figure out, how to easily import the parsed results to
D... Hmmh, I might have an idea; if I modify that DMDFE to output the
parse tree in some kind of ASCII stream, I might be able to construct
it in D; hmmh, that sounds somewhat silly... Well, I try to figure out
something. Again, thanks for that link! :D

No need to go ascii. Just wrap up the data in some structs and add some
extern(C) functions that you can call from D which return those structs.

Yes, I thought that, too, but I thought that it could be quite an effort.
Maybe later :) I thought that if I put the dmdfe dumping info from
parsing, the same can be used for importing the tree to D. But I'll play
around for awhile to find a suitable way of doing it.

Or if you're doing D2.0, then there's that extern(C++) thing that may
work.

Does it work with g++? I haven't yet used D2.0, but maybe it is time to
try it out, too :)

Hi,
I thought that I could play around a little bit with DMD parser. I have
DMD 1.030 parser sources, and I made a simple makefile for just compiling
those sources (with gcc 4.3.1).
---
Some questions:
- These are C++ sources, not C sources, although the suffix is .c and .h,
right? A little bit sad, I thought that I could wrap the parser to D :( I
have not much interest playing with C++ at freetime...
- Are these files meant to be compiled alone without making modifications
to files? For example:
- In "attrib.c", if linux is defined, it tries to include mem.h from "../
root"; why not the one in the src/ directory?
- There is no such file that "id.h", which probably contains class/struct/
enum "Id"
- init.h does not include "hdrgen.h", while uses HdrGenState
---
How these parser codes are meant to be used? Am I missing something?

The DParser project on dsource may be of interest too, it is a D port of the
parsing parts of the DMD frontend.
--
Lars Ivar Igesund
blog at http://larsivi.net
DSource, #d.tango & #D: larsivi
Dancing the Tango

Hi,
I thought that I could play around a little bit with DMD parser. I have
DMD 1.030 parser sources, and I made a simple makefile for just compiling
those sources (with gcc 4.3.1).
---
Some questions:
- These are C++ sources, not C sources, although the suffix is .c and .h,
right? A little bit sad, I thought that I could wrap the parser to D :( I
have not much interest playing with C++ at freetime...
- Are these files meant to be compiled alone without making modifications
to files? For example:
- In "attrib.c", if linux is defined, it tries to include mem.h from "../
root"; why not the one in the src/ directory?
- There is no such file that "id.h", which probably contains
class/struct/
enum "Id"
- init.h does not include "hdrgen.h", while uses HdrGenState
---
How these parser codes are meant to be used? Am I missing something?

I'm working on something that could be of interest to you.
I have a working DMD to D bridge via C abi. It's not complete but most
important data structures are available from D, i.e. you can pass some .d
files to it and get an AST back. I've written a small source-code analyzer
as a proof-of-concept with it.
Think of it as a DParser (http://www.dsource.org/projects/dparser) but
with official implementation. A huge plus is that almost no maintaining
effort needed.
Bridge is written from scratch with a mix of C++ and D1 and is not based
on DMDFE (http://www.dsource.org/projects/dmdfe). I haven't released the
sources yet, so contact me if you would like to look at the them.

Yes, I was already wondering, how DParser will keep up-to-date. It would
be a huge effort to go through the new C++ files and check, what need to
be changed in DParser...

Bridge is written from scratch with a mix of C++ and D1 and is not based
on DMDFE (http://www.dsource.org/projects/dmdfe). I haven't released the
sources yet, so contact me if you would like to look at the them.

Hi,
I thought that I could play around a little bit with DMD parser. I have
DMD 1.030 parser sources, and I made a simple makefile for just compiling
those sources (with gcc 4.3.1).
---
Some questions:
- These are C++ sources, not C sources, although the suffix is .c and .h,
right? A little bit sad, I thought that I could wrap the parser to D :( I
have not much interest playing with C++ at freetime...
- Are these files meant to be compiled alone without making modifications
to files? For example:
- In "attrib.c", if linux is defined, it tries to include mem.h from "../
root"; why not the one in the src/ directory?
- There is no such file that "id.h", which probably contains
class/struct/
enum "Id"
- init.h does not include "hdrgen.h", while uses HdrGenState
---
How these parser codes are meant to be used? Am I missing something?

actually dparser is not 1.014 anymore. it's somewhat 1.024ish
the parsing ability has been improved a lot, it can parse itself, and some
ctfes.
700 or so cases from dstress fail.
--
使用 Opera 革命性的电子邮件客户程序: http://www.opera.com/mail/

Hi,
I thought that I could play around a little bit with DMD parser. I have
DMD 1.030 parser sources, and I made a simple makefile for just compiling
those sources (with gcc 4.3.1).
---
Some questions:
- These are C++ sources, not C sources, although the suffix is .c and .h,
right? A little bit sad, I thought that I could wrap the parser to D :( I
have not much interest playing with C++ at freetime...
- Are these files meant to be compiled alone without making modifications
to files? For example:
- In "attrib.c", if linux is defined, it tries to include mem.h from "../
root"; why not the one in the src/ directory?
- There is no such file that "id.h", which probably contains
class/struct/
enum "Id"
- init.h does not include "hdrgen.h", while uses HdrGenState
---
How these parser codes are meant to be used? Am I missing something?

actually dparser is not 1.014 anymore. it's somewhat 1.024ish
the parsing ability has been improved a lot, it can parse itself, and
some ctfes.
700 or so cases from dstress fail.

I hope you don't mind that I too the liberty of updating the dparser
wiki page to reflect that information.
--bb

actually dparser is not 1.014 anymore. it's somewhat 1.024ish the
parsing ability has been improved a lot, it can parse itself, and some
ctfes.
700 or so cases from dstress fail.

:o
I fetched the sources (I tried to select the latest ones), and attached
that DParser to my own code. I copied the set-ups from trunk/d1.0/dparser/
dmd/main.d, and do parsing and semantic passes 1 to 3 to build the parsed
tree.
Problems:
- Parser segfaults, if it parses std.stdio
- I'm trying to write a tree walker; I have walk(X) functions for
different types of elements (Module, FuncDeclaration,
DeclStatement, ...). But I have hard times to determine the types of
statements, since Statement class contains only a few "isXXXX()"
functions. I tried to use typeid(typeof(s)), but it returns - as you may
guess - always the type that was declared in the function arguments.
---
I use DMD1.030, and I have those 1.030 phobos sources.
I'll try to figure out how to extract that statement type information
from parse tree, and could wait for newer versions of DParser.
At the same time I'll try to get dil compiled; although dil does not
generate code, it might still be a suitable platform to write a software
to generate some common warnings (just for my personal use, I'm not
intended to start to write "dlint"). Unfortunately it means that I need
to figure out how those DSSS & Tango things work :)

Hi,
I thought that I could play around a little bit with DMD parser. I have
DMD 1.030 parser sources, and I made a simple makefile for just
compiling
those sources (with gcc 4.3.1).
---
Some questions:
- These are C++ sources, not C sources, although the suffix is .c and
.h,
right? A little bit sad, I thought that I could wrap the parser to D
:( I
have not much interest playing with C++ at freetime...
- Are these files meant to be compiled alone without making
modifications
to files? For example:
- In "attrib.c", if linux is defined, it tries to include mem.h from
"../
root"; why not the one in the src/ directory?
- There is no such file that "id.h", which probably contains
class/struct/
enum "Id"
- init.h does not include "hdrgen.h", while uses HdrGenState
---
How these parser codes are meant to be used? Am I missing something?

the parsing ability has been improved a lot, it can parse itself, and
some ctfes.
700 or so cases from dstress fail.

I hope you don't mind that I too the liberty of updating the dparser
wiki page to reflect that information.
--bb

actually dparser is not 1.014 anymore. it's somewhat 1.024ish the
parsing ability has been improved a lot, it can parse itself, and some
ctfes.
700 or so cases from dstress fail.

:o
I fetched the sources (I tried to select the latest ones), and attached
that DParser to my own code. I copied the set-ups from
trunk/d1.0/dparser/
dmd/main.d, and do parsing and semantic passes 1 to 3 to build the parsed
tree.
Problems:
- Parser segfaults, if it parses std.stdio
- I'm trying to write a tree walker; I have walk(X) functions for
different types of elements (Module, FuncDeclaration,
DeclStatement, ...). But I have hard times to determine the types of
statements, since Statement class contains only a few "isXXXX()"
functions. I tried to use typeid(typeof(s)), but it returns - as you may
guess - always the type that was declared in the function arguments.
---
I use DMD1.030, and I have those 1.030 phobos sources.
I'll try to figure out how to extract that statement type information
from parse tree, and could wait for newer versions of DParser.
At the same time I'll try to get dil compiled; although dil does not
generate code, it might still be a suitable platform to write a software
to generate some common warnings (just for my personal use, I'm not
intended to start to write "dlint"). Unfortunately it means that I need
to figure out how those DSSS & Tango things work :)

err, i didn't maintain the trunk/d1.0/dparser/dmd/main.d
and surely the d2.0 branch also not maintained at the moment.
use d1.0/test_dparser
It shouldn't segfault. std.stdio has been parsed a lot of times here :)
It's a good start from there I think
--
使用 Opera 革命性的电子邮件客户程序: http://www.opera.com/mail/

err, i didn't maintain the trunk/d1.0/dparser/dmd/main.d and surely the
d2.0 branch also not maintained at the moment.
use d1.0/test_dparser
It shouldn't segfault. std.stdio has been parsed a lot of times here :)
It's a good start from there I think

Yes, that worked :)
Sorry to use unmaintained things, but there is such a little information
written in that package :) I just took first file I saw containing main,
after I had written a somewhat functional Makefile :)
But that does not solve my problem to determine, which kind of statement
type there is in the tree; I can currently extract only those, which have
isXXXX functions in the Statement class.

At the same time I'll try to get dil compiled; although dil does not
generate code, it might still be a suitable platform to write a software
to generate some common warnings (just for my personal use, I'm not
intended to start to write "dlint"). Unfortunately it means that I need
to figure out how those DSSS & Tango things work :)

I finally made it, but that was extremely hard :D Now dil compiles \o/
I'm not sure where to tell these findings, so I write them here;
hopefully someone finds them with google, if he/she enters same
problems...
DSSS, GDC, Tango and Linux-AMD64:
---
First thing to know is that currently GDC (nor DMD) does not work
completely with 64-bit Linuxes. Everything else seems to work, but not
vararg's (that is, writef & writefln).
So, install a chroot'ed 32-bit environment, instructions:
http://saftsack.fs.uni-bayreuth.de/~dun3/archives/using-a-x86-chroot-to-
run-32-bit-apps-on-debian-amd64/94.html
---
You may install DMD, it does not cause any harm :)
---
Second thing; if you followed those instructions, Debian sid comes with
newer GCC than is supported by GDC. So don't install it (it may work, but
better to use correct version)! Install first GDC (currently GDC-4.1),
and install the same version of GCC and G++ (4.1).
Try to compile a simple hello.d with GDC to check, that it really works.
With correct versions of all necessary things (gcc, g++, gdc and their
libraries & utils), gdc should work OK.
---
Next thing to install is DSSS. It should work also, quite easily. Make it
use GDC (to be able to build tango with GDC). I was not able to install
curl to chrooted sid, so I'm not able to use dsss net installs.
---
When you have GDC and DSSS working, you can try to get tango working. I
fought with tango for several hours. I don't really know what phase
didn't work, but here are something:
* I tried both dsss build/install, and build/install scripts in tango
package (currently 0.99.6). Both seemed to work (no errors), but GDC
didn't build my simple tango-hello.d; there were continuously error
messages like:
hello.o: In function `_Dmain':
hello.d:(.text+0x20): undefined reference to
`_D5tango2io7Console4CoutC5tango2io7Console7Console6Output'
hello.d:(.text+0x2f): undefined reference to
`_D5tango2io7Console7Console6Output6appendMFAaZC5tango2io7Console7Console6Output'
hello.d:(.text+0x3b): undefined reference to
`_D5tango2io7Console7Console6Output7newlineMFZC5tango2io7Console7Console6Output'
hello.o:(.data+0x38): undefined reference to
`_D5tango2io7Console12__ModuleInfoZ'
There were probably two reasons for this:
1) libgphobos.a in /usr/lib/gcc/i486-linux-gnu/4.1.3/ was probably
incorrectly build, and I didn't notice to update it.
2) It seems, that libDG-tango-core.a does not include all the necessary
files. I ran "ar r libDG-tango-core.a *.o" in tango/ directory (where the
libDG* files are), since there was object files like
tango.core.Exception.o), which were not in that core library.
---
* In tango directory, use script:
./build-tango.sh gdc
...To build libgtango.a, which you copy to correct location (good guess: /
usr/lib/gcc/i486-linux-gnu/4.1.3/)
* It could be easier first to try to get plain gdc builds to work:
$ gdmd -version=Posix tango-hello.d -ofhello -W-lgtango
---
Now when I got those things working, I'm not going to uninstall them and
try again, so I can't tell step-by-step instructions for getting them
working ;D

Oh, it's pretty awesome you get that far in such a short time.
I bet you can print instance.classinfo.name to get the type name of that
instance.
And I want to have your code be integrated into the dparser trunk for
illustrating
how to use dparser.
--
使用 Opera 革命性的电子邮件客户程序: http://www.opera.com/mail/

I'm wondering how to extract the type of those statements... At least
they are not any of the Statement.isXXXX -types.

Oh, it's pretty awesome you get that far in such a short time. I bet you
can print instance.classinfo.name to get the type name of that instance.

Hmmh, I'll try that... I'm still playing around to get used to these new
D things (dsss, tango, tango-phobos -coexistence etc)...

And I want to have your code be integrated into the dparser trunk for
illustrating how to use dparser.

I needed to make some modifications to dparser sources to get them
compiled to Linux (if I remeber correctly). I'll try to do that walking
thing, and then I get fresh, unmodified dparser and see, if I get it
compiled without any hacking.

Of course!
Send me the patch? If you get a dsource account, maybe you're the right
guy to cooperate on this interesting project, I may grant you write access
;)
--
使用 Opera 革命性的电子邮件客户程序: http://www.opera.com/mail/

I'll put the full sources available somewhere (to my home pages, at
least), and continue developing walking & warning thing.

That would be very useful.

I'm currently hacking more scanners:
- Scanning/warning, if function return value is discarded, i.e.
int f() { ... }
f(); // Warn
cast(void)f(); // Don't warn
- Warning for suspicious looking infinite loops, i.e.
for(;3.1415926535894;) { ... } // Warn
for(;;) { } // Don't warn
There are some problems to make them correct, since DParser does not
always store all necessary information, e.g. "for(;true;)" = "for(;;)"
I probably include scanning of constant if/switch values with the loop
scanning... That general purpose walking class makes implementation of
these kinds of checkings relatively easy.
There will surely missing / wrongly working pieces in the code, but those
can be corrected later :)

The unused variable bug has been appeared in the bugzilla, and yes, i did
think about it a bit.
The way I think is modify the VarDeclaration to have a member flag to show
this var is used or not.
But this require a more complicated way of coding, your walker might be
the better way to code. If
we can balance the performance issue, I think we open something really
interesting in balancing of
design pattern and performance.
--
使用 Opera 革命性的电子邮件客户程序: http://www.opera.com/mail/

I'll put the full sources available somewhere (to my home pages, at
least), and continue developing walking & warning thing.

Sources are available:
http://reaaliaika.net/showarea.php/download/dwalk
dwalk.tar.ge opens to directory named dwalk. dwalk/dparser should contain
link to DParser sources (you may also upload them to dwalk/ directory
with svn etc).
Package contains:
- Sources: dwalk/src/
- Test files: dwalk/testfiles
About sources: You probably first take a look to main.d. Then you may
want to look at the scanners (ScanXXXModule.d). Finally, you might want
to take a look at the ASTWalk (ASTWalking.d).
If you get the thing compiled, you should have executable named dwalk,
which can:
- Detect unused things; there may still be some false reporting, as well
as it can surely undetect some unused things. imports are not checked at
all. Nor templates (or they may be, if they go with the defaults).
- Detect expression result discarding, i.e. unused return values. There
are some pecularities with "void main(...)" functions, which cause false
alarms.
- Detect constant conditions, in for/while/do and if/switch; take a look
at the testfiles/TestInfLoop1.d for seeing, which kind of infinite or
dead loops are scanned.
Any reports, comments & suggestions are welcome :D

Oh, some additions... Packet includes binary for i386-linux. Packet
compiles with DMD1.0 and phobos. If compiling to linux, you may need to
hack (current) DParser sources a little; if I remember correctly, change
memicmp -> memcmp somewhere (changes things case sensitive, bu so what),
and somewhere else there was function like DebugOutputA, change it to
writefln.