In fact, I would love to hear ideas of how you might think you would use JDBC in any kind of game on any scale.

Of a 2-3 year development cycle game we've been releasing this spring/summer, we used several thousand of ini files to maintain parameters like game balance, AI control values, animation control, etc.While this worked, the balance and data manager guys didn't have an easy job (neither did we have). We learned that next time we would use a database (maybe a file datbase like dBase would do).

However, to use datbases within Win32 is not very nice, neither. Compared to the ease and elegance of JDCB. So on a similar project in Java I would use JDBC.(The step from painfully E-SQL C on Unix to JDBC Java on a mainframe has been a dream :-)

So you are saying the game configuration/customization is in 1000's of external files, that could be game run-time accessed as JDBC records?

Yes.

Furthermore: during the development cycle it would be nice if the developers could alter the database cells at game runtime, and write back the changes.So the designers, artists, etc could do an ideal game balance / animation control at runtime. With external files this isn't a nice and secure tasks (no table/row locks, etc).

Quote

Like for example all RPG character and stats perhaps?

For example, yes.With a nice design of 1..n tables a DB approach is handier, more robust and more efficient. We had to use tons of GAWK scripts to maintain these 1000's of external files. Not again. ;-)

For a stand alone game I think JDBC is overkill ressource and performance wise. You would need to distribute a database, the JDBC driver and the whole set of JDBC classes. Just too big.For that I would recommend some small OO database with speed in mind and some minor restrictions compared to "real world" databases. Examples are Prevayler (www.prevayler.org) or Smyle (www.drjava.de/smyle) which are a few thousand times faster than JDBC against Oracle or MySQL.You should never forget that JDBC implies mapping records from database tables to Java objects and vice versa. Those conversions take time, are error prone and always imply code refactoring if you change tables or classes...

Only for huge server applications (MMORPGs for example) I could imagine storing client data (accounts, character information) and world data in some database on the server.But under all circumstances I would try to avoid distributing JDBC to the client side.

Just the 2 cent of a day by day JDBC and server app coder and wannabe game coder...Tommy

For a stand alone game I think JDBC is overkill ressource and performance wise. You would need to distribute a database, the JDBC driver and the whole set of JDBC classes. Just too big.

At game runtime just reading the file database should be sufficient. Maybe using the built-in J2SE JDCB-ODBC bridge driver to read some file database like dBase would do.

Quote

You should never forget that JDBC implies mapping records from database tables to Java objects and vice versa. Those conversions take time, are error prone and always imply code refactoring if you change tables or classes...

Not neccessary Java objects. Simple types like resultset.getInt(..), .getFloat(..), .getBoolean(..) and that like could be OK. Main point: to avoid 1000's of external files to control your animation, AI etc. However this data cells have to be editiable during development by non-developers, too.

Since these data cells are being read at game startup and then cached time shoudn't be an issue.

Hi I'm currently planning for larger scale versions of my game, i'm no where near implementing the plans, but it's in my nogin. Any data that is shiped across to the client over a network, when being persisted (at backup/logout times) will be done with JDBC or the likes, this includes the account information, characters and all the information on stats location, sector, friends lists etc etc, data only of course, i'm thinking a table per class probably, and store just the data. I've done smaller tests using xml and it's ok for levels etc that are stored on the client, but for persisted data nothing beats a *good* database (they have spent more time on performance than I have on high performance file reading ). To start with I'm thinking postgresql/MySQL (as I have them running already). Even for medium sized data storage (in game terms) like score tables in non persistant worlds it makes some sence.

Endolf

P.S. I'm not sure where that post was going, but I thought i'd throw in my 2p worth as it were

I agree with shawnkendall. An ad hoc solution can be tough, introducing bugs, etc. I'd say it's safer (more cost-effective?) to leverage something like JDBC, which is a proven and mature technology. JDBC has a great track record, running in mission critical enterprise software.

Also, using a database is MUCH FASTER than accessing flat files! And if you need, JDBC can access flat files, too. ;-)

<opinion>Tbh, I've yet to see any java game (of appropriate scale) get to state where using JDBC would make sense. (maybe Magicosm or Wurm would be the first).

For trivial things things using JDBC may be slight overkill with the setup and maintainance of the database and the memory overhead of actually bring up a JDBC driver (from what I remember, the implementations I've used MySQL, SQLServer don't seem to be that light).

Although I suppose *IF* you're happy to expose your database to the outside world it does stop you having to write a high-score server for client only games. However, if you've only got trivial data to store and you're going to have a server anyway I'm not its really worthwhile.</opinion>

Although I suppose *IF* you're happy to expose your database to the outside world it does stop you having to write a high-score server for client only games.

Yes if you use JDBC, wouldn't you then be forced to put the DB login information in the game? Feels like from a security perspective it would be easier to make a server. Or maybe a server that sits between the DB and the user. A DB driven highscore list for example would require the user client to be able to both read and write to the DB, which would make it quiet easy to cheat. The highscore is probably a bad example though, since it pretty easy to cheat anyway . Lets take an example in an persistent rpg, what would stop the user from adding skills to the DB if the user client talks directly to the DB?

In near future, we are planning on using JDBC or something functional similiar Here is our reasoning.

1) During content production the content production tools use the database for asset management. It makes it easier to back-up and restore if need be, and we can extend any existing file formats with meta data that is game important, as oNyx suggests.

2) For large 3D worlds that would otherwise need a custom paging database system away (continuous worlds, no level loading) so why not use JDBC instead.

3) Massive character and item attribute data is simply a stupid pain in the *** to make 100s (1000s?) files and directory management for and even more cumbersome to prototype with. Of course we can make it One Big File (tm) but then we have to be ready to make an access system that can handle efficient data inserts, which means the One Big File would have to have a page system just like disks anyway.Using a database allows a bit more developmental and design freedom.

4) At GDC last year one of the lectures I went really struct a cord. The speaker said that when creting your content storage design (file format) make sure you can import, export, and edit portions of the final content and not have to re-export the entire asset set. Using a db really solves this problem as well as handling the whole multiple people in one file/area issues and if the run-time engine can access the exact same db without conversion then we really have a nice system :-)

Does it sound like JDBC would still be to "heavy-weight" in this context? I definitely want to identify weaknesses and counter reasons.

I think the caveat on the negative posts above is simply scale. You've outlined points that refer to fairly huge scale projects/development. If we're talking 1000's of elements of data then the logical choice is always going to be a database (even if its file based like hsql). However, most of the projects round these parts arn't actually on that scale..

I know we'd all like to build truely scalable systems but there is an overhead involved in using JDBC (runtime and dev-time). However, the trade off between using and not using is going to be based on the personal experience of the developer. If I'd used JDBC everyday of my life for 10 years (impossible?) then even for the most trivial game its going to be worth thinking about.

JDBC is a damn good way of storing data reliably with no fuss - the techniques are well known and you can pick up any number of books on the subject. Also, most Java developers in the world today are J2EE developers, who already know JDBC.

The biggest problem with regards games is that J2EE API implementations aren't exactly optimised for either speed or memory consumption! Or at least not for reducing memory consumption...

But as a quick and easy way of persisting state, reference-counting, handling crashes, sorting objects, querying object trees etc, it's hard to beat a DB. JDBC is simple and really easy to integrate with an application. You can write your ultra-fast low-memory DB implementation and swap it in at a later date if required.

Also, using a database is MUCH FASTER than accessing flat files! And if you need, JDBC can access flat files, too. ;-)

That's not true for many moderate sized data sets (and for small datasets DB's are usually particularly slow). You can arrange your flat files to be extremely fast, and feature-rich DB's can never do more than be "no worse than flat files". The major reason for using DB's is that you get many advanced features WITHOUT losing speed - you don't get an increase in speed with a DB if your original data architecture was good.

Really big DB's are a separate issue - e.g. when they are large enough that a non-DB implementation has to work around the hardware architecture and such like.

In near future, we are planning on using JDBC or something functional similiar Here is our reasoning.

On the client-side, your proposal is a good one. We do something similar (in a parallel situation; our clients are servers) but we have a complex software layer on top of the JDBC stuff - I suppose similar to a complex 3D engine on top of OpenGL.

In our situation, the database stuff is used for extremely rapid development of game-logic (and we've got a patent pending on some of the layer above which saves devs a lot of time), and then we either snapshot that for distribution or allow clients direct read-only access - but we work on MMOG's, so distribution is to servers in a cluster rather than home PC clients.

Quote

1) During content production the content production tools use the database for asset management.

Frankly, anyone who isn't doing this already can't claim to be a serious, professional game development studio. For anyone to whom this is not obvious, there are actually 3rd party tools that have been around for a long time that are specifically marketd to games devs for this kind of stuff.

...but you still occasionally see post-mortems where it wasn't done!

Quote

2) For large 3D worlds that would otherwise need a custom paging database system away (continuous worlds, no level loading) so why not use JDBC instead.

This is a situation we do have, and have found that (unsurprisingly) JDBC is fine but needs a heck of a lot of work on top of it (c.f. analogy to 3D engine and OpenGL above; it's pretty obvious, but I'm also trying to give an idea of how much code and dev needs to go into making that layer work...it's not a trivial affair).

Quote

3) Massive character and item attribute data is simply a stupid pain in the *** to make 100s (1000s?) to prototype with.

Yup, as I said this is an area where we've done a lot of R&D. We were only going to license that part of our tech, but it reduces the dev time and cost by so much we're now investigating developing and publishing our own games! (greatly reduced risk...)

Quote

Does it sound like JDBC would still be to "heavy-weight" in this context? I definitely want to identify weaknesses and counter reasons.

As an extension to all the server-focussed distribution of "development code as DB" above, we've done some very vague investigation of distributing single-player games with a DB - very much as you describe, except that we would be interested in re-leveraging our DB layer for single-player game dev, so in our case it's less of a case "should we do this?" than of "we've already got this; what problems will crop up to make it hard to use in clientside code?".

The only major problems we saw were:

- fast DB's cost serious mahola to distribute to your end-users. Unfortunately the likes of Oracle have still not experienced a shakeup of their industry, don't work too hard as a company, and haven't really cottoned on to allowing cheap (preferably free) distribution of a read-only runtime or similar for this kind of thing.

- MySQL is outpaced by a blind snail tied to a ten-tonne weight. Even "OR" was not allowed to use indexes up to at least 4.x (I cannot IIRC whether 4 fixed this yet). There are no stored procs (!) etc. Give MySQL another 3 years and it should be quite fast.

However, that's from *our* perspective; if I understand correctly, you're only really looking at using it for simple data, and could use the snapshot approach like us (so that clients don't need a copy of the dB software to read the data), in which case I see almost no problem at all...

PS JDBC is IME not at all heavyweight. Poor DB performance and/or people who don't know how to write SQL queries properly completely disguises the overhead of JDBC. If you've got people with decent DBA skill you should be fine.

I know we'd all like to build truely scalable systems but there is an overhead involved in using JDBC (runtime and dev-time). However, the trade off between using and not using is going to be based on the personal experience of the developer. If I'd used JDBC everyday of my life for 10 years (impossible?) then even for the most trivial game its going to be worth thinking about.

I'm not sure how you see scalablility of games having much to do with JDBC usage/avoidance?

JDBC (or, more importantly, SQL) is very easy to use, and it takes only a few months max for even a novice developer to be writing "good" SQL (manual optimization is still a VERY big issue in DB work...).

I would suggest that it's worth learning basic SQL just because it's so incredibly simple (note: IIRC it was one of those designed to be readable by illiterate managers back in the 70's..?). Just look at how many incompetent programmers easily manage to write big PHP websites with tonnes of SQL!

I'm not sure how you see scalablility of games having much to do with JDBC usage/avoidance?

I guess I was coming at it from a hobbiest point of view, if you don't already know SQL (which as it happens, I do) and you're about to store a trivial amount of data then its probably not worth you going through the (possibly minor) hassle of learning it and setting up the DB.

However, if you expect to scale your game later on to need a hell of a lot more data then its worth the up front work. Thats all.

For a stand alone game I think JDBC is overkill ressource and performance wise. You would need to distribute a database, the JDBC driver and the whole set of JDBC classes. Just too big.

There are some VERY small pure Java embedded JDBC database implementations. PointBase for instance.

The biggest issue for a game IMO is cost. Due to the complexity of implementing efficient SQL handling, JDBC databases don't tend to be cheap. The two exceptions of course are mysql and PostgreSQL. These however are not small and don't embed.

Frankly, for a stand-alone game I'd look at something small and simple like JDBM (available on sourceforge.)

Got a question about Java and game programming? Just new to the Java Game Development Community? Try my FAQ. Its likely you'll learn something!

We're using JDBC on both the client and server side for Galactic Village, which is an MMO game.

On the server side, we've used Sybase, Oracle, and Postgres (and soon MySql).

On the client side we've used Sybase, Postgres and CloudScape. I'm going to be looking at PointBase and MySql for the client as well, but would also be interested in other lightweight, Java-based client-side databases.

We've thought about using XML-JDBC drivers to let us read/write XML files (for editing tools and stuff), although I haven't seen a good, free XML-JDBC driver that I like. Plus we'd have to create an XSD from our database schema, which takes time and effort.

We wrote our own persistence management layer, which is JDO-ish but not JDO-compliant (yet), and looks kinda like bean-managed persistence in EJBs . This has enabled us do true database-swapping, even with some of the databases not being totally ANSI-SQL compliant or having annoying things like names folding to lower-case in Postgres.

I've thought about using EJBs, but there's overhead that we don't need, so we're not going to switch to it at this time.

Shawn, I'd be happy to go into more detail if you want to reference Galactic Village in your book! ;-)

BTW, at CGDC we'll be showing server-side technology that happens to be implemented using JDBC to talk to a VERY fast (and not cheap) database system. (TimesTen for anyone interested, a quick google should give you their website.)

I do recommend however others do what we did, which was *first* define what we needed in terms of data access functionality and built an Interface. THEN looked at technologies we could implement it with below that "firebreak." Thus we are not tied only to databases that implement JDBC but can use any technology that meets our (much smaller) functional and (highly intense) performance requirements.

IMO by the time you are done tuning you end up having to do so much system-specific SQL for performance reasons that even using JDBC your code still ends up DB specific. So make your architectural break a level higher.

Got a question about Java and game programming? Just new to the Java Game Development Community? Try my FAQ. Its likely you'll learn something!

BTW, at CGDC we'll be showing server-side technology that happens to be implemented using JDBC

Is this "we" the GTG or Sun generally? I.e. is it something that you guys have put together, or are you promoting an internal project from another part of the organization?

Quote

IMO by the time you are done tuning you end up having to do so much system-specific SQL for performance reasons that even using JDBC your code still ends up DB specific. So make your architectural break a level higher.

I'd just like to triple-underline this statement . For all the things SQL manages to achieve, being a cross-vendor standard is not, in reality, one of them . As Jeff says, as soon as you are done tuning (heck, IME almost the moment you START tuning!), you can kiss goodbye to vendor independence.

IIRC I've been told this is mainly due to the extreme slowness of the standards process (always the case...) and inadequate extension mechanisms - so most vendors had to add lots of proprietary language features just to keep their software up to date for customer needs. But I'm not a DBA, and I'm just repeating what I've heard...

To go in another direction: If you're going to do something large-scale, networked/distributed, then you might want to leverage EJB technology as well. Entity EJB's wrap (basically implementation of DAO pattern) the database using using JDBC at the lower levels, and you don't have to write SQL or JDBC code. The EJB container manages reads/writes for you, and can even manage transactions.

JBoss is a free J2EE environment and comes with okay EJB support, although I'd suggest using WebLogic if you had the money. Not sure what is quality enough for MMOG. In any case, EJB's give you a managed environment. Either that, or check out JMS. JMS introduces the concept of MBeans (managed beans). Point is, for something large-scale, you need container management, proven scalability and reliability, and all that you might find in the J2EE (fault-tolerance, fail-over, clustering).

Not sure what is quality enough for MMOG. In any case, EJB's give you a managed environment. Either that, or check out JMS. JMS introduces the concept of MBeans (managed beans). Point is, for something large-scale, you need container management, proven scalability and reliability, and all that you might find in the J2EE (fault-tolerance, fail-over, clustering).

Bearing in mind that most of the servers (such as Weblogic, JBoss - even Oracle - etc) are much more than "only a j2ee implementation" (so may be great for MMOG's but you have to examine each on a case-by-case basis):

On the whole, J2EE sucks for MMOG development. From an MMOG perspective, it's slow, clunky, full of useless clutter, and ill-equipped to cope with the needs of MMOG development.

This is merely becasue J2EE was designed for a completely different purpose (i.e. there's nothing particularly slow about J2EE in the environment it was intended for); Majestic, for instance, was much more like an enterprise app than an MMOG (IMHO this has a lot to do with why it failed commercially!) so J2EE was not a bad choice in that situation, technically.

I don't view DB as something usefull in most games. My game, for example, would need 200 TB if it would use DB. Without DB it could be 24 GB (hopefully).

Also DB will not decrease development time, if you'd use custom development applications that would have part of game code inside.More compex games need custom data manipulation, so aside savegame conversions there isn't much what would create big problems in the next version. (if made) I consider lack of documentation as bigger problem than a outdated custom data format.

java-gaming.org is not responsible for the content posted by its members, including references to external websites,
and other references that may or may not have a relation with our primarily
gaming and game production oriented community.
inquiries and complaints can be sent via email to the info‑account of the
company managing the website of java‑gaming.org