Programming games in Java ... is a bit like making spaceships out of Lego.

Just one of those strange little analogies that pops into one's head now again. Remember what making spaceships out of Lego was like in the mid-70s? All your spaceships were rather square, and opaque, and didn't have any lasers or antennae. But the wise and benevolent Lego company realised that we needed some special-purpose bricks to make our spaceships especially sleek. So we got lasers, and little spacemen with oxygen tanks and helmets, and ariels, and thrusters, and wing-shaped bits, and computer panels, and transparent screens...

Right now I feel like I'm using Lego in the early 70s when writing in Java! But if you all go and vote for structs[/b][/i] we'll have a rocket thruster piece to play with! It'll still be square and laserless but we can get those bits for Christmas! Vote vote vote! Even if you wouldn't know a struct from a fruitcake you'll be helping us all!

I have just finished reading the RFE you posted on Structs. I similiarly was pushing for a "struct" data object in Java last year.

However, the mechanism you suggest I strongly disagree with (at least as much as I understoof it from the RFE).

Specifically, requiring the struct to have a buffer backend. This effective makes a single struct instance even larger in memory, and one of the benefits of a struct should be less memory than a equivalent Java class. (IMHO)

What you are describing is a bytebuffer wrapping mechanism. "Real" java structs do not (should not) be anywhere but in Java memory space. One of the HUGE gain for have structs in Java is much better memory use for large arrays and other collections of struct objects (i.e. Matrices). In addition, Java structs shoudl be methodless and non-inheritable - i.e. c-like struct, for absolute minimum memory footprint and maxium performance. The closest thing to this now in Java is simple arrays of primitives, but that is a very non-OO way to code up things and is very ugly syntax. (currently how we do 3D character animation)

Please give more comments, I would love to hammer out a complete view of this stuff publicly now. (although you on this list are damn quick, much quickly than I get to respond it seems!)

I too have been hammering on about structs in Java since the boards started over 2 years ago but it was a bit like arguing with a brick wall :-/

The mechanism I proposed is therefore a cunning twist on the simple struct-as-primitive-type idea. It's cunning because it fits in really nicely with NIO, which is where I see structs being most useful (for me, anyway).

There is indeed an overhead with using an object to represent a single struct but I've cunningly designed the mechanism to re-use instances. And indeed without a reference type it's going to be pretty inefficient passing structs about to Java methods.

Also, I can't really see anyone accepting that a struct can't have methods associated with it.

So you see, what I've got here is a compromise which is likely to solve most of the problems we've got and fit in nicely with what we already have.

You could still, for example, have an exceedingly space efficient BSP tree implemented as structs. Pointers to nodes are simple stored as int buffer offsets; you only need 1 instance of BSPNode to walk the tree. It takes up basically the same memory as in C. Nice or what?

The mechanism I proposed is therefore a cunning twist on the simple struct-as-primitive-type idea. It's cunning because it fits in really nicely with NIO, which is where I see structs being most useful (for me, anyway).

No offense but if it's a cunning twist on structs, maybe it's not "structs"

Quote

There is indeed an overhead with using an object to represent a single struct but I've cunningly designed the mechanism to re-use instances. And indeed without a reference type it's going to be pretty inefficient passing structs about to Java methods.

I see no reason why the structs I explained are passable by reference, in fact, they must be just like objects.I'm not sure I understand how you re-use instances...

Quote

Also, I can't really see anyone accepting that a struct can't have methods associated with it.

I disagree. That is what a struct is, a compact, non-inheritable, methodless object. Structs in C do not have object methods, they are functions that can operate on those structs. In OO that would be in a different class. The typical example is a Vector struct, and Vector utils class that processes Vector structs.

Quote

So you see, what I've got here is a compromise which is likely to solve most of the problems we've got and fit in nicely with what we already have.

Well if it is a compromise, then I suggest calling it something different. A struct has very specific meaning and more importantly expectations for the user of them.

What you have described as I understand it is some native access improvement. Structs can and should be a pure Java concept, and are agnostic to native access.

In all seriuosness, the idea of a struct ( a compact, non-inheritable, methodless object ) is a great addition to the Java Language, that I would like to see.This is not what you are describing.

Now that "old, old, idea from an ancient language called C" is still in existence, and more importantly here, in heavy use in gaming libraries, for example RenderWare, the biggest money producing game engine out today.Besides is existing use, it is a good technique (dare I say "best") to use for many, many problems. But, unfortunately, what you are describing is something different. You have to have some respect for your fellow programmers, and except the working definition of "Struct". You can't simply reinvent what that means and expect it to be accepted by the existing community.

Maybe there is some ByteBuffer name derivative that would sound better(sometimes names are that critical to acceptable and avoiding confusion).

Or another possibility would be a Struct type, and then a NativeStruct can mapped directly to native buffers. But here there still would be no methods or inheritance, like the C-sde buffer and C-struct. I hate to have yet another layer for the MappedObjects,i.e MappedObjects layer NativeStructs that wrap ByteBuffers. This would maintain "struct" functionality independant of native buffers AND independant of java objects, as well as allow your MappedObject idea, but it certainly seems like too much.

How can they be seperate and both have the mapped native access you want, where on is a real Java object and the other is the c-struct-like objectless type?

I still think Struct is probably the most accurate name for this feature. After all, the notion of a Java array is almost completely different from the C version and no-one's complained. Much.

I spent long and hard trying to figure out how you could actually implement pure C structs in the Java language and concluded there would be some problems.

To implement a struct you have to make a decision on how it behaves. Either it behaves like a primitive type, or it behaves like a reference type.

If it's a primitive type, we have to pass-by-value to method calls, which will involve possibly large amounts of data copying. Furthermore you are then restricted in that you cannot have circular struct references in definitions as there is no pointer type in Java. So there goes BSPs, linked lists, quadtrees, etc.

If it's a reference type then it will need to descend from Object and be garbage collected like anything else, so the whole memory advantage is blown again. However it then fits seamlessly into Java's method dispatching system etc. and will allow it to have methods associated with it. Again, I can't see a good reason not to have methods associated with them - it's one of the best bits about OO programming, and there's no reason to throw the baby out with the bathwater. And finally of course if you've got a native buffer filled with native C-structs you still can't manipulate that buffer directly, it needs to be read into objects and copied back again.

I could in fact see only one way which we could get the memory efficiency of a C-struct, the semantics of a Java Object, and very close to the memory bandwidth efficiency of using C-structs, without breaking the VM too much.

That's to restrict the use of structs to storage in ByteBuffers - think about it, it's the only way to avoid the Object overhead - and map their fields using JVM trickery direct to the memory of the buffer. A struct effectively becomes a pointer to a bit of memory in the buffer - just like in C - but it's a proper Java Object so we'll call it a Struct and not a struct.

This means that a Struct can't contain Object derived fields, only primitives - as there is no semantic for mapping arbitraty Objects to bytes in a buffer! - but it would be reasonable to allow other Structs inside a Struct. What you still can't do is store a pointer to another Struct - but instead you'd use an int offset into the buffer (measured in Struct sizes, not bytes). The whole mechanism is so beautifully simple and elegant, but above all, it's super-efficient, totally safe, and easy to implement. I'm quite pleased with myself for thinking of it

There really isn't any place a natural C-struct fits in the Java language or the VM, you see. To get the RFE through the hurdles it's got to appeal the VM engineers and language ****s, and they will never buy the idea of implementing C-structs in Java because of the aforementioned wrangles.

Stucts in Java is against Java's nature. Objects should do in all cases. If you need something faster/more-flexible/more-reliable you should be using c/c++ by now (and don't you dare flaming me just because I'm right). Pushing to its limits a language who's programs are interpreted, makes no sense at all.

" If it's a reference type then it will need to descend from Object and be garbage collected like anything else, so the whole memory advantage is blown again"

I think you are missing the memory advantage I want. I don't care that it is GCed, in fact, I 'm counting on it.

I don't care about Native memory ( I do, but for agrument sake, I don't) so why should be Struct have anything to do with ByteBuffers. Now if the VM wants to map it that way behind the scenes, I could careless.

What I want/need, is a c-struct Struct, simple, clean, efficient.

Yes, you can build trees with structs, they can have pointers, so Java Structs can have object references.

Yes, they should be passed by reference and not by copy (this one is obvious - this isn't LISP :-))

No, they do not need methods or inheritances, and that IS the point of the object I am describing. Structs hold data (primitives and references) and nothing else. If mapping them to buffers limits their references use, forget it. I think about it, that will never get pass the VM/Language people - a new object-liek type that can't reference the existing Java objects except through user implemented indirection tables!? Whoa :-)

The idea here is to make a Java object type that would work like Vectors and Martices do in C, because it is the lowest possible memory overhead and run-time overhead, and for certain programming techniques is cleaner than straight OO (ok last one is opinion :-))

The thing you are desrcibing now has too many exceptions (no Java references) and misses the run-time gains (no method reflection overhead, JIT, etc.)

The computation math folks have been discussing similiar contructs since Java ran on vector processors, because the overhead when you have 100,000s of vectors and matrices (also in 3D) ends up blowing your memory and run-time access performance.

One more wrench I want to toss in there since we are "dreaming" - I want a way I can declare arrays of your structs or my structs created in such a way that they are contiguous memory, like in C/C++, that way the caching behavior of sequential access is much better :-) (since that is in high use in the struct techinques involving vectors etc - although I imagine a VM can do this behind the scenes, at least for "fresh" arrays)

Stucts in Java is against Java's nature. Objects should do in all cases. If you need something faster/more-flexible/more-reliable you should be using c/c++ by now (and don't you dare flaming me just because I'm right). Pushing to its limits a language who's programs are interpreted, makes no sense at all.

One, Java does have assemble access right now, through the JNI. We will be using it to access P4 SIMD vector instructions for batch 3D stuff.

Two, Java is NOT interpreted on all platforms, namely J2SE, it's JITed so the most running parts are native executible.

Three, there is not reason that "Struct" type is not an Object as well, it just a Data Object, not unlike the Java Data Object API, or any relational databases use.

And you can't really say if you want something else go somewhere else, because java is evolving based on user and industry input, and it BETTER evolve, or .Net languages will over take it. C# allows very similiar features to what we are saying here, and it is the fastest growing language right now - [source Slashdot links]

Here, we are just trying to get clear a solution for significant problems we face using Java.

It's no argument at all. I don't think cmr here has ever tried to do anything like what we do in Java. We know it can be done but it's a pain in the arse right now, and 50% of the speed it could be. I merely want to make it easier, and faster, without a) making life hard for the VM engineers and b) scaring off newbies. Only people who know why Structs are necessary need to use Structs. This ain't like dustin' crops, boy.

Besides, the solution I'm suggesting is incredibly simple and not a massive change to the JVM.

Shawn's suggestion is rather more complex and I think you've missed some critical timesavers in your idea. We've got a neat way to get contiguous areas of RAM to store structs in already - bytebuffers!

And also, there's no way you can have references to structs in Java, and hope to get them garbage collected, without the same overhead as a normal Object. It would simply be far too difficult to figure out, let alone implement; but deriving Struct from Object is a no-brainer. Just point an instance at an address in a buffer and bingo - all the properties of both an Object and a C struct. The most useful advantage of all this - as someone who's really pushing the boundaries of Java here, for sure - is that I get access to native side rendering structures for free as well!. With pure Java native structs, you'd still be forced to write them into a buffer after you'd fiddled with them, or devise something that looked uncannily like my Struct implementation over StructBuffers.

Hmmm, doing some usage test cases.Have to get back to ya. :-)Two things doing it my way gets rid of is Class overhead (the objects aren't that big), which is the JIT and reflection overhead for each inherited type. That is a bloater and with arguable usefulness in a struct. The features you want are nice and interesting (auto-typing/casting, copyless native access), but I just don't think we are talking about solving the same problems.

I DO respect the fact that you both try to improve this beautiful language (absolutely no irony/sarcasm here), but I also believe that you are trying to make a VW beatle into a Mercedes by changing the windshield.

I'll ask you a very simple question and I'm outta your faces: If there was a magical way, to instantly transform your current projects into C/C++/the-graphics-lib-of-your-choice(you're making games, right?) ones, would you do it? Be honest.

What I'm trying to say is that Java provides convenience from which you cannot easily escape-but you should when you reach the level that you guys are at.

And Cas baby, I might not be doing what you're doing now, but I survived the 80s using machine code, I spent the 90s with c and c++ and currently I'm working on a new 3D engine for a games studio in London. Java is my hobby so I am a Java newbie compared to you, but I kinda know what I'm talking about.

I wish I could just say "Wow" to this last post, but I'm noisier than that. :-)

Quote

I DO respect the fact that you both try to improve this beautiful language (absolutely no irony/sarcasm here), but I also believe that you are trying to make a VW beatle into a Mercedes by changing the windshield.

I'll ask you a very simple question and I'm outta your faces: If there was a magical way, to instantly transform your current projects into C/C++/the-graphics-lib-of-your-choice(you're making games, right?) ones, would you do it? Be honest.

Maybe if you also gave me a C/C++ compiler that made my all my code as written run on Win and Linux, did GC, didn't crash on null pointer exceptions (as well as division by zero, and numberous other crash sources), actually used exceptions, and was secure...(for starters)

Quote

What I'm trying to say is that Java provides convenience from which you cannot easily escape-but you should when you reach the level that you guys are at.

And Cas baby, I might not be doing what you're doing now, but I survived the 80s using machine code, I spent the 90s with c and c++ and currently I'm working on a new 3D engine for a games studio in London. Java is my hobby so I am a Java newbie compared to you, but I kinda know what I'm talking about.

Good luck to both of you

When people are having a discussion to clarify a piece of technology, why would you jump in and argue whether we SHOULD be having the discussion, because for you it is a forgone conclusion that the entire technology base is inadequate. To take an assertively anti-progressive position towards other's activities in a public forum, well, is your right, but foreign to me.After this post, I can only think to myself, why are you at this forum? Carefully disguised flamebait is still flamebait.I guess I have to stop reponding to this now, and get on with what we were trying to work out.

The more I think about it, the only time I would really want a struct is when interfacing to a specific file structure, or native structure. I'm not bothered (yet?) by the extra memory required for java objects. The main inconvenience is the manual packing/unpacking for IO and access to native data structures.

So I tend to side more with princec on this, because it seems the least disruptive to the the language.

This could be simply because I haven't encountered an example that requires structs for anything other than memory effiency, and that memory savings alone doesn't seem to be a good enough reason for true structs. Again, maybe I'm just missing the right example.

The thing is, my design also solves the memory efficiency issue as well - and it does it without resorting to pointers, and it's memory safe. You can store a BSP tree in a ByteBuffer and recurse through it using only one instance of a BSPNode. Nice. And the bonus of having methods on it too What more could a programmer ask for?

Well it doesn't solve the memory issues in a way that fits in with how you would want to use structs in Java... (basically the same as objects).I wouldn't want a separate ByteBuffer backing each struct, and it is irritating to have to pass around an index or something if you want to pass an individual struct from a buffer containing many... but for IO and native interface purposes it is a reasonable compromise.

Sure thing, I never wanted to use Structs in that way anyway - I've got Objects to do that, and they do it better than C++ does it. It's a win-win situation! Vote vote vote! We only need another 20 odd and it'll be on the top 25!

java-gaming.org is not responsible for the content posted by its members, including references to external websites,
and other references that may or may not have a relation with our primarily
gaming and game production oriented community.
inquiries and complaints can be sent via email to the info‑account of the
company managing the website of java‑gaming.org