This has been discussed a couple of times, and at the moment I'm in the middle of writing a web-based game which relies heavily on anything that reduces dev time (if I had more time, I'd be working on JGF!). XML has come up several times as an option for file-format, and I've finally switched to it for *some* config files. I've used XML before on lots of projects like this, using everything from full custom SAX decoders and schemas, through bastardisations of Sun's XMLDecoder, and even using XMLDecoder "pure". I've looked back at the old threads on here, and none seemed to come to much in the way of concrete conclusions, so I thought I'd start an article on my experiences and ideas, doing this for the N'th time

At the moment, I'm still just jotting down ideas and issues as I explore the different options. More sections will appear on this page over the next few weeks, then I'll write it all up. Of course, keep your fingers crossed that JGF happens to be up at the time (I'm still having to reboot the server once a day ).

As many of you are aware I'm a big hater of the "XML for Everything" movement, but for info like this which is hriearchically orineted, has to be half-way human readable/writable and/or is going to get pumped across text-oriented bridge like HTTP, and where wriite/read performance isnt of primary importance, it CAN be an (ugly) time-saver.

I still say the real meaning of XML is eXcessively Messy Languages and that a custom data labguage would almost always be a neater more elegent solution, but it CAN be a useful short-cut when you "just don't care enough to send the very best."

Got a question about Java and game programming? Just new to the Java Game Development Community? Try my FAQ. Its likely you'll learn something!

I used XML in a previous version of my game (well, one day will be a game, I hope ) but hated it. Easy to make mistakes if I edit it with a text editor, difficult to read, lack of a fixed DTD (because I keep adding properties and classes to my engine) so I couldn't validate, etc. What's the advantage here? So one day I changed it all to properties files and what a difference!

Quote

...at which point, Sun's Properties object becomes entirely useless

Really? I have had no problem with hierarchical properties. Just implemented a subclass of Properties with two modifications:

-The ability of include a properties file within another (so my config files can be well structured and remain small enough to be easy to read and edit manually), this can be made scanning the file prior to loading it or simply establishing a special property name, in my case, every one that ends in ".include".

-A collapse function that scans all the keys and extract the ones that start with a given prefix, then construct a hashtable<String, MyProperties> like this:

After collapsing I can iterate over the keyset (names of the "objects") and get the properties of each one. I find it really easy and intuitive to use. Perhaps the collapse method isn't very optimized but I only use it at loading time, so...

I used XML in a previous version of my game (well, one day will be a game, I hope ) but hated it. Easy to make mistakes if I edit it with a text editor, difficult to read, lack of a fixed DTD (because I keep adding properties and classes to my engine) so I couldn't validate, etc. What's the advantage here? So one day I changed it all to properties files and what a difference!

sounds like your structure of your xml is wrong.

properties files are quite usefull for simple keysets, take internationalisation for example.

xml for configuration well overall I have good experiences, but I'ves encountered mostly wel structured well setup examples. I do have seen very bad, I'd almost say abused xml.

Could be, of course. But I don't know how a game that extends some components from my 2D engine (i.e., a new Text that looks like a comic book speech balloon) can be validated from the engine without providing a new DTD that includes (by hand, I assume, don't think you can include a DTD inside another, perhaps with schemas... I'm not a XML guru by any means, I fear) all the standards components and the new ones... And I don't think it is really worth the extra effort. If someone can show me a different approach to validating, please, post.

Besides that (and knowing that a simple and obvios solution can exist), I find much more easier to use my new classes and iterate over arrays and read from hash tables than to navigate between nodes of an DOM tree (and I have done both things. ); and think that a properties file is much more human-readable (and human-editable!) than XML, both being valuable advantages, from my point of view.

But I'm not trying to persuade anyone, of course, just showing some alternatives that perhaps can spark some ideas...

I love a bit of rapid development myself. For one project I wanted the user to be able to configure the project with XML. But for the system to be neat you really need all the options to have an object representation. Parsing the XML and building an object representation is tedious, especially if the XML structure keeps changing and invalidating your parse code. The solution I came up was:-

1. Pick a single package for your object representation to reside in. Fill it full of very simple objects that can be connected into a heirachy and also give them String attributes for things you would like the user to configure.

2.Create a reflective parser. Basically this just recurses the DOM or SAX and for each Element it sees it instanciates a class whose package name is (1). The String attributes of the element are copied into memebers of the new object of the same name*. The difficult bit is deciding how you connect the heirachy up. In my non-generic case things were fairly simple and it was enough to either :- add a child object to a parents attribute of type collection<ChildObjectType> iff one existed OR assign the child object to a varable of type ChildObject if one existed.

Obviously this approach is not fully flexable for all possible XML structures, but the ones it does not fit are probably poorly designed. The great thing is though that the workflow for adding options is just to add new attributes to your object representations or create new classes in the package.

PS Actually I did have to intergrate my parsing stratergy with an independantly developed parser for some elements of my XML. To make your XML fully configurable I created a ParserConfig. This was a big HashTable of ElementNames -> ParserObjects along with a default. The parser config then gets to decide which parser handles what element types. In my case the defualt parser was the reflective parser. This alowed me to write my special cases for Element types that were a bit tricky in some respect.

* PPS actually I also had a bit of intelligent parsing here. If the target attribute was an int, then Integer.parseInt was called on the XML attribute first to assign it.

Runesketch: an Online CCG built on Google App Engine where players draw their cards and trade. Fight, draw or trade yourself to success.

uuurgh.I hate XML with a passion. It all SEEMS like such a good idea. And then you have to trawl through the specs for JBI, Web services, and just to throw a spanner into the works, BPEL (or rather BPEL4WS as we haven't moved to version 2 yet). Using the PXE BPEL engine you have to write ...1. an XML WSDL file to expose the actual BPEL service itself,2. The BPEL process, which is a state based language with procedural elements and is the ugliest misabuse of XML I've ever seen,3. The BPEL deployment descriptor, also in XML4. An extra XML file for each of the WSDL services you're planning on using in the BPEL process, to expose their partnerLinks

All of these,needless to say, are completely interdependent, with various element names you have to drag from a variety of different depths in each others hierarchies and use correctly in another file. And did I mention the namespaces ? Hundreds of them. HUNDREDS ! and woe betide if you get a single thing wrong ...

-ahem- Okay, thats my little OT rant over,

back ON topic, I've used (thankfully) much simpler XML scripts for quite a while for configuration of things like models, shaders, animation effects and so on. Something like the following:

How did you avoid writing a parser? How did the values in the DOM model or SAX or whatever get into your java objects? At some level you would have needed a parser surely....

okay, so minimal code then. What I meant by a parser in the OP was having to parse the file itself, getting tokens in and REing them to get values etc etc. The various XML libraries make this bit redundant.

How did you avoid writing a parser? How did the values in the DOM model or SAX or whatever get into your java objects? At some level you would have needed a parser surely....

I thin kthe point is that the parser is a standard component, thanks yto all XMl docs sharing a common syntax. There are quite a few pacakges that will proiduce complete parsers for you from a DTD

Having said that, I think this is a VASTLY overrated feature.

Automatic parser generation has been a reality, ina much more flexible form, since *I* was in college (and thatw as abck in teh days whenw e wrote flow charts on cave walls) through YACC and Lex. I have a stadnard parser I wrote quite awhile ago that has "expect..." and "get..." calls and using it I can write a parser for any basic text data language in a few lines of code.

Got a question about Java and game programming? Just new to the Java Game Development Community? Try my FAQ. Its likely you'll learn something!

Oh wow. That Xstream is good. goodbye ReflectiveParser, the same idea but only partially implemented!

Quote

Automatic parser generation has been a reality, ina much more flexible form, since *I* was in college (and thatw as abck in teh days whenw e wrote flow charts on cave walls) through YACC and Lex. I have a stadnard parser I wrote quite awhile ago that has "expect..." and "get..." calls and using it I can write a parser for any basic text data language in a few lines of code.

Yeah. I know that. Yacc etc though still need configuration. I wanted a solution that worked with no configuration or programming and that would update itself when you change all your model classes.

Runesketch: an Online CCG built on Google App Engine where players draw their cards and trade. Fight, draw or trade yourself to success.

Automatic parser generation has been a reality, ina much more flexible form, since *I* was in college (and thatw as abck in teh days whenw e wrote flow charts on cave walls) through YACC and Lex. I have a stadnard parser I wrote quite awhile ago that has "expect..." and "get..." calls and using it I can write a parser for any basic text data language in a few lines of code.

Sure, but YACC still isn't properly documented (at least it wasn't circa 12 months ago last time I looked, still just says basically "if you already know how to write grammers, you can use this, if not, go learn") and is insanely difficult to use for non-academic purposes .

NB: this is speaking as someone who has used all this suff in the past, written grammars, parsers, and parser-generators, etc. Its just that when you haven't done it for years, you know it's trivial, but the bloody software is written with not a single thought for making it usable - it punishes you for not still having memorized all the domain-specific jargon (which has no purpose being in a tool, which is what yacc etc should be).

So, yeah, YACC and friends can do it all. But ... the XML stuff is aimed at real people, so unsuprisingly is a heck of a lot more popular . SAX, bar knowing two gotcha (to do with the chars() method buffering, and with whitespace reading), is trivial to use and you can use it immediately with no special knowledge. That is how life should be .

Having to write a grammer though totally gets in the way of productivity. You have to then maintain that grammer in situations where you saving format is changing all the time. Plus grammers in general are normally pretty verbose. Also YACC is not java ....

Anyway, so I like the whole reflection method of parsing XML. You don't need to maintain anything apart from your model which you would have to do anyway.

Runesketch: an Online CCG built on Google App Engine where players draw their cards and trade. Fight, draw or trade yourself to success.

Having to write a grammer though totally gets in the way of productivity. You have to then maintain that grammer in situations where you saving format is changing all the time.

And this is different from a DTD or hand coded XML reading how?

Quote

Plus grammers in general are normally pretty verbose.

Actually I 100% disagree. A well written gramer describes a LOT of code extremely compactly.

And if verbosity bothers you,a well designed app specific data language will be MUCH less verbose then eXessively Messy Languages.

Quote

Also YACC is not java ....

YACC was mentioned for historical reasons. There are at least half a dozen Java parser generators out there.

Quote

Anyway, so I like the whole reflection method of parsing XML. You don't need to maintain anything apart from your model which you would have to do anyway.

All this is is a fixed mapping for Java object to XML. Given a fixed mapping to any other data format you could do exactly the same thing.

But that fixed mapping almost ensures that the XML form will be highly inconvenient for hand writing/editing as it isn't beign driven by the descriptive needs of the app domain. At which point why bother with XML at all? Just serialize. Its a lot more efficient.

Got a question about Java and game programming? Just new to the Java Game Development Community? Try my FAQ. Its likely you'll learn something!

I was quoting the use of reflective parsers. So there is no hand writing of XML reading.

Quote

All this is is a fixed mapping for Java object to XML.

yes

Quote

At which point why bother with XML at all?

Becauce you can edit the XML

Quote

But that fixed mapping almost ensures that the XML form will be highly inconvenient for hand writing/editing as it isn't beign driven by the descriptive needs of the app domain.

No, your classnames and variables just have to be good enough to present to your user. You will have to make sure this part of your code base IS driven by the app domain. True, you can argue that this is crossing the developer-user barrier. But if it saves you development time then who cares? Your options are to either1 write your object model with app driven names and not write a parser or2 write your object model with arbitary names, then write a parser than converts between the two.

The first option requires you to write just a single object model. The second requires you to write an object model of exactly the same size, but also with a verbose parser that invloves alot of trival assignments.

Quote

And if verbosity bothers you,a well designed app specific data language will be MUCH less verbose then eXessively Messy Languages.

I am not crazy on XML. But I do like it that my java editor supports XML, and can even do intelligent code completion on my XML (yes! even without a DTD! (when I used to use intelliJ)). Your custom developed data formats cannot do that, so it more typing for you.

Runesketch: an Online CCG built on Google App Engine where players draw their cards and trade. Fight, draw or trade yourself to success.

Mostly fair enough. Though again I find the chocies auto-created formats make to be at bear minimum annopying and inconvenient. YMMV ofcourse.

I have to raise an eyebrow at the editing point. You *can* edit XML, but the fact of the matter is, accoridng to my brother who was on the W3C DOM comittee, XMl is specificly *not* designed for hand editing.

This is why it has significant whitespace, something you *really* want to avoid in any human written language.

Got a question about Java and game programming? Just new to the Java Game Development Community? Try my FAQ. Its likely you'll learn something!

Actually I 100% disagree. A well written gramer describes a LOT of code extremely compactly.

Um, yeah, me too - there's pretty much no point at all to a grammar, practically speaking, if it is NOT extremely terse. That's what it's there for!

I am hugely in favour of grammars - I believe that everyone who understands them will agree they are always a better solution than not having them - but my problem is that the tools for using them today generally suck so badly they are worse than nothing at all. They totally undo the advantages of the grammar itself .

OK. I will retract that. I spose the only stuff I ever worked with was parsing proper languages, and those grammars are big but then they are as complex as it gets.

Quote

I believe that everyone who understands them will agree they are always a better solution than not having them

Normally if I find a problem that needs one, I try and change the problem. Like using BSH instead of writing my own scripting language or what have you. I just find writing grammars and stuff fiddly and time consuming, though admitadly I have only really done it 3 times in my life (C compiler, Java 1.4 and a 1.5 parser).

Quote

I have to raise an eyebrow at the editing point. You *can* edit XML, but the fact of the matter is, accoridng to my brother who was on the W3C DOM comittee, XMl is specificly *not* designed for hand editing.

Thats interesting. I never knew that. But we are game developers so to hell with the rules! Haha.

Quote

but my problem is that the tools for using them today generally suck so badly they are worse than nothing at all.

Actually I think this is the crux of the matter really. I only like using XML in my back end becuase its intergrated with my development environment and I get intelligent editing with it. Also for one of my projects the user had to configure the system, so I decided do it in XML becuase then the user can use their intelligent editor of choice with it. So it all comes down to the wealth of third party tools you get with XML (for me). In theory there is no reason why you couldn't have similar stuff for grammars, but the tools don't exist and its too fiddly as a result.

Runesketch: an Online CCG built on Google App Engine where players draw their cards and trade. Fight, draw or trade yourself to success.

java-gaming.org is not responsible for the content posted by its members, including references to external websites,
and other references that may or may not have a relation with our primarily
gaming and game production oriented community.
inquiries and complaints can be sent via email to the info‑account of the
company managing the website of java‑gaming.org