The idea for researching this is to see if there are differences between the file types in such a way that would not allow a conversion process to take place to make a G3M file be a G1M file and work on the older calculators. Would make project producing for multiple models much easier, if they could be built for all 3 types in one go. Especially if one isn't using color items through the G3M file type.

Edit:

Going to go ahead and paste this here, since Jonimus shared it on IRC:

Last edited by KermMartian on 29 Jul 2014 11:39:41 am; edited 1 time in total

Differences:
- Byte @0x08: $CE in the G1M, $8A in the G3M.
- Byte @0x15: $00 in the G1M, $01 in the G3M.
- Word @0x1C: $00,$00 in the G1M, $49,$2C in the G3M.

Notes based on my SourceCoder 3 code to generate g3m files:
- Bytes at 0x0E, 0x13, 0x14 are part of file integrity/security. The fact that these are identical means that this part of the "security" between the two types is also identical.
- Bytes 0x1C, 0x1D are also part of file integrity/security. In the g1m these are 0s; in the g3m these are based on the size of the file through known equations:
-- *0x1D = 0x38 - *0x48 (g3m only)
-- *0x0E = 0x72 - *0x48
-- *0x14 = *0x1D + 0xC3
-- *0x13 = *0x14 - 0x48
-- *0x1C = *0x13 - 0x5E (g3m only)
- The data starting at 0x3C is the 8-character, 0-padded name (0x41 == 'A')

Edit: Actually, the bytes at 0x10 to 0x13 are the 4-byte big-endian size integer of the entire file (not just the program data), with every bit inverted. The *0x13 value indicated above is therefore rather silly; what it actually means is that *0x14 = ~(LSB of size integer) + 0x48. The values at 0x1C and 0x1D can also be re-phrased as a function of the value at 0x13.

Would it be possible for you to dump a lot of symbols into a .g3m program, I (or you) can hex-edit it to have a .g1m header instead, and you can try loading it on an older calculator? I'd be curious to see what the older calculator would do with a program with symbols it theoretically doesn't know.

I made an attempt to do as you asked, and the linking program wouldn't do anything with it. It's possible I didn't change the correct things, though, so I could email you the unchanged file if you want to muddle with it to send back.

Thanks, I appreciate the effort. I'll need to learn more before I try to craft a file that I think will definitely send. In the meantime, I've been trying to crack the more challenging problem of g3p files. I'm quite confident by now that they're a compressed format of some sort; I have the following excerpts from my extensive notes to share so far:

Code:

- Header similar to g3p format
- 0x0e, 0x0f, 0x12, and 0x15 may include security values
- Where programs have the string "PROGRAM" (type) and "system" (folder) in the header, pictures have "CP0100Ly755" and no folder
- The size of the data is repeated many times, in gradually-decreasing forms. It appears that the format actually has several layers wrapping the previous layers, like an onion.
- The data itself appears to be 0x15B bytes smaller than the file itself.

KermMartian, IIRC there is lots of info on g3p files at Omnimaga, and on Simon's docs there is information for some inflate/deflate syscalls; consensus some years ago was to stop reverse-engineering the g3p file format to avoid having Casio lock the platform as TI did with the Nspire.

And of course it is a compressed format, more specifically, it uses gzip compression. The OS includes zlib. The g3p file format also accepts 8-color (3 bit) images for screenshots taken when the screen is in this color mode. They are considerably smaller, but since they don't have a fixed size, we can assume they are compressed too.

On Simon's docs it says that file formats written by the Prizm share a common header format, and then each format has a different subheader. Meaning of different header fields can be found in his docs.

Also note that the g3m is a common container for all main-memory (MCS) backups, not just programs. If you directly change a g3m to a g1m, there may be problems due to possible MCS changes between calculator versions, like folders changing name and the files in MCS changing format. So, even if you manage to produce a vaild g1m from a g3m, the program format the older calculator expects may be different, causing problems unrelated to the file format conversion.

The fact that you see multiple file sizes on the g3p header may be due to the fact that it is on the main header, and then a smaller value (i.e. the data minus the header) certainly repeated on the subheader, giving the "onion" appearance.

Yes, I have uncovered a number of very interesting facts about the g3p format, many of which I am not publishing for now (including the exact compression algorithm and the obfuscation method Casio uses). I will be performing some PHP and Python experiments later to verify the accuracy of my discoveries, but I am aware of the moral implications. I was one of the community leaders who objected when Simon first started working with creating an image converter and patching the OS to make all cg10 and cg20 models load the same pictures. With the introduction of the TI-84+CSE, I have rethought my position a bit, so if I get something working, I will open up the issue to Cemetech discussion before making any decision on publishing it.

Making the CG10 load any g3p image is not so much about patching the OS (which is only necessary if you don't have a CG10 to test on), but about patching the file... and last time I checked, it was as simple as changing some bits and recalculating the checksum.

Also note that any "feedback" from Casio may not have any effect in the current generation of calculators, but in the next one. The reasons that make me believe this, are:
a) the Prizm appears to have reached the EOD (end-of-development) phase, with OS 2.00 adding a bunch of new features they were "cooking" for some time, and any updates from on now will probably only fix bugs.
b) locking down the platform by enforcing e.g. code signing would break compatibility with their own add-ins, which would mean they would need to release new versions of all official add-ins and get people to update to them. This needs some engineering and leads us to the next point;
c) locking down the OS at this point would require programmers to focus on the Prizm, when at this point the engineering force is probably already focused on developing the next iteration of calculators. Such major changes to the OS would also require lots of testing, and the testing efforts are probably better spent testing the new models.

As mentionned in the SourceCoder 3 topic and several times in the past three years before, it would also be nice to investigate on that other g3p image file format that seems pretty much ignored by every FX-cg10 developers in the TI and Casio community. I know that most people focus on the one discussed above that supports 65536 colors and is restricted to European calculator models, but there is also a second format that is FX-cg10-compatible that supports 8 colors and in some cases is smaller.

As mentioned in the SourceCoder 3 topic and several times in the past three years before, it would also be nice to investigate on that other g3p image file format that seems pretty much ignored by every FX-cg10 developers in the TI and Casio community. I know that most people focus on the one discussed above that supports 65536 colors and is restricted to European calculator models, but there is also a second format that is FX-cg10-compatible that supports 8 colors and in some cases is smaller.

The files that you gave me and that I am investigating are structurally extremely similar to the 16-bit-color images, and I believe understanding one would make understanding both trivial. I will update this topic once I have results from tonight's experiments.

Cool to hear. I am definitively curious about how compression works, because some simpler files are smaller than others, unlike on the CSE where every 8xi file is 21 KB. If there is a color dithering pattern that allows images to be smaller, then maybe dithering options could be implemented in an image converter to allow the creation of such image.

Cool to hear. I am definitively curious about how compression works, because some simpler files are smaller than others, unlike on the CSE where every 8xi file is 21 KB. If there is a color dithering pattern that allows images to be smaller, then maybe dithering options could be implemented in an image converter to allow the creation of such image.

I wonder if compression is RLE-based?

No, the compression is not RLE-based. It is in fact based on a well-established compression algorithm that works well for arbitrary data; gbl08ma gave a hint as to what it is. Casio added a small amount of obfuscation over it to make it less obvious what the algorithm is, but I believe I have figured out how to reverse that obfuscation.

I know that most people focus on the one discussed above that supports 65536 colors and is restricted to European calculator models, but there is also a second format that is FX-cg10-compatible that supports 8 colors and in some cases is smaller.

No, no, no. You are misunderstanding it.
Both the CG 10 and the CG 20 can read 65536 color and 8 color images.
The CG 10 can only read g3p in which the "Provided by Casio" bit is set (regardless of the color depth), regardless of the color depth. Proof: try to take a screenshot of the main menu[1], or a picture in my JPEG reader[2], on a CG 10. There you have your 65536 color image on a CG 10.
The CG 20 can read any g3p regardless of the "Provided by Casio" bit or the color depth.

[1] the situation for screenshots may be a bit different; I'm not sure if screenshots created on a CG 10 are compatible with all the Prizms, or only with the calculator where they were created - it's possible that the header contains the unique board ID (8 bytes at 0x8001FFD0, not set on emulator) and that the CG 10 checks that before displaying the image.
[2] used by many as a g3p converter that creates CG 10-compatible g3p files

Yeah I know that the cg10 can read 16 bit images in Casio's format. Sorry for not clarifying. What I meant is that it didn't support third-party images of that format, only the ones made by Casio. I didn't realize that this was the same thing with 3 bit images, though, considering I could create them on-calc.

COuld anybody try to open the g3p files I posted a few days ago on their FX-cg10 to see if they work? If not, then I guess this could definitively be a problem.

I am curious about whether they can open those as well. I will need fx-CG10 and fx-CG20 users to check whether the images I manage to create work on both sets of calculators, for that matter, now that I'm starting to pull together code to generate images. I'm trying to side-step the issue of computing the convoluted footers, because it looks like it might just be easier to generate "Provided by Casio" formats.

The Geometry and V-Window data is not mandatory; the Geometry add-in can open such files just fine, users only need to answer yes to the question about setting initial values, or something like that. And for the remaining uses of g3p files, I think there's no difference. You only need to find a way to cleanly avoid setting such data and you are done (unless you want to go so far as to reverse-engineer the V-Window and Geometry data formats).

I made an attempt to do as you asked, and the linking program wouldn't do anything with it. It's possible I didn't change the correct things, though, so I could email you the unchanged file if you want to muddle with it to send back.

Also, datestamp for g3m files: 10/1/2010 12:00 AM

The timestamp thing appears to be an issue with how the Prizm transfers programs as a USB device, unfortunately. Would you mind emailing me that modified program so I can poke at the header myself? If the 9750 does not actively reject files with symbols/tokens it can't render, I will not make a new token file for it, and simply rely on programmers to recognize when they've used something the calculator can't handle.

Have your own thoughts to add to this or any other topic? Want to ask a question, offer a suggestion, share your own programs and projects, upload a file to the file archives, get help with calculator and computer programming, or simply chat with like-minded coders and tech and calculator enthusiasts via the site-wide AJAX SAX widget? Registration for a free Cemetech account only takes a minute.