Version 1.0009/18/06Written by K.HortonThanks to Quietust for ideas and proofing and help----

This is the tentative addition to the standard .NES file format thatmost emulators use. This addition is designed to disambiguate certainROMs that currently can only be discerned via a CRC-32 or similar hashcheck. Naturally, this causes problems for new ROMs that are not in thedatabase, but need special handling.

There are four goals for this specification.

1) Retain 100% backwards compatibility with existing emulators/ROMs/etc. (*this includes "dirty ROMs" with crap such as "diskdude!!!" in the header and other atrocities*)

2) The format must be "future proof".

3) The changes made must be VERY CAREFULLY documented and make sense.

4) Said changes must make sense from both a hardware and software standpoint.

N: Upper 4 bits of the mapper numberP: Playchoice 10. When set, this is a PC-10 gameV: Vs. Unisystem. When set, this is a Vs. gamex: these bits are not used.

* * *

For NES 2.0, none of the above will be changing, except for the two unusedbits on flags byte 1.

To indicate that this is a NES 2.0 file, bit 2 will be clear, and bit 3 willbe set. i.e.

7 0---------NNNN 10PV

Same as above, but the 1 and 0 pattern will denote an NES 2.0 file. Thisneatly gets around the "diskdude!!!" problem, because those headers do nothave the correct bits set on this byte, and will thus be treated as aregular iNES file.

That is the "how" of determining whether we are dealing with a valid NES 2.0file. Now that that is done, the desired aspects of such a thing need tobe considered. I have tested over 4000 ROMs and have dumped at least athousand more, and reverse engineered probably 60-70 different mappers.This has given me a front row seat into the shortcomings of the original,and a good idea of where additional information is needed.

* * *

The new things we need to know are thusly:

1) Vs. Unisystem

The Vs. Unisystem is one of the two Nintendo arcade machine series producedwhich use "mostly NES/famicom" hardware. These games will run fine onemulators if a little extra things are stored in the header.

Nintendo wanted to make life difficult for arcade operators when it came tocopy protection. Three main schemes were devised. (See the "Vs. system byte"description below for a detailed analysis)

2) PRG ROM in excess of 2Mbytes, CHR ROM in excess of 1Mbyte

This has already occured, and has been causing trouble for some ROMs. Sofar, the hack has been to set PRG ROM to 00h to indicate 4Mbytes of ROM(since FFh is 16K short of 4Mbytes), and in the case of exceeding the2Mbyte-8K CHR barrier, ROMs have been allocating the CHR in the PRG space,and the emulator has to sort this out. Very messy.

Currently, the only fix for this is to CRC the games and then hack the mapperif the CRC or other hash matches. This of course fails if the game is notin the database.

4) Mapper numbers

Face it, we're running out of mapper numbers. 16 seemed like it would beenough, but they were quickly exhausted. Then 256 mappers seemed like avast space to work on. But now, we are getting near the end of the line,and running out of mapper numbers. I have personally assigned at least 50or 60 of them, which is almost 1/4th of the total mapper space.

5) WRAM

Not all carts that support WRAM support 8K of it. Some support less,some support more, and some even have EEPROM! Heck, some carts evenbattery backed the stupid CHR RAM. This last one was a very recentfind and goes to show that a workable extention needs to reasonablycover all possible bases.

* * *

The proposed solution:

Byte 8:

7 0---------SSSS xxxM

S: Sub-mapper number.

This specifies the submapper for this ROM. If no submapper mode isneeded, set this nybble to 0000b.

M: Mapper number extension.

CAUTION: DO NOT USE THIS BIT YET. There are still some existingnumbers left in the current iNES mapper space. (Around 30-40 or so bymy last count). The other three bits marked with "x" are also slatedto be used if more than 512 mappers are required. This would allowsupport for 4096 mappers all together. This should hold us until thenext iceage. I repeat: do not designate mappers above 256 yet.

--

Byte 9:

7 0---------CCCC PPPP

C: 4 more CHR ROM size bitsP: 4 more PRG ROM size bits

These combine with the existing 8 bits of each to form 12 bits totalfor the number of PRG and CHR banks... this is enough for 64Mbytes-16Kof PRG data and 32Mbytes-8K of CHR data.

--

For the following two bytes, this table defines the size of the RAMsegments:

p: Quantity of PRG RAM which is battery backed (or serial EEPROM, see below)P: Quantity of PRG RAM which is NOT battery backed

--

Byte 11:7 0---------cccc CCCC

c: Quantity of CHR RAM which is battery backed (yes it exists! see below)C: Quantity of CHR RAM which is NOT battery backed

--

A note about serial EEPROMs and battery backed CHR RAM...Some mapper 16 (Bandai) games use serial EEPROMs to store the game data,rather than a battery backed SRAM. These can be as small as 128 bytes oras large as 512 bytes. They tended to use 24C01 (128 bytes) 24C02 (256bytes) or another semicustom chip I cannot find a workalike for. Theinterface for the 24Cxx parts is I^2E which is a Philips specification.The workalike chip is very similar, but the address and data are clockedin backwards from the I^2C parts.

As for battery backed CHR RAM, the Racermate cartridge has 64K of CHR RAMtotal: 32K is battery backed, and 32K of it is not. They store all thestats and such in it. Why you would do such a thing, I do not have aclue... but they did! I traced out the circuit myself and couldn'tbelieve it.

--

Byte 12:7 0---------xxxx xxBP

P: This is a PAL ROM. When set, indicates PAL mode.B: When set, indicates this ROM works on both PAL and NTSC machines.Some of the Codemasters games actually will adjust the game dependingon if it detects you running on a PAL or NTSC machine - it adjusts thetiming of the game, and fixes the music.

Not many games would have this B flag set.

x: These bits are not used yet. They shall be maintained clear.

byte 13:7 0---------MMMM PPPP

This byte is reserved for the Vs. Unisystem only. If this is not a Vs.Unisystem ROM, then this byte shall be all 0's.

I have dumped the palettes from ALL of these PPUs, and have exact bit forbit copies of them. The last 5 PPUs (RC2C05) have the standard NESpalette in them, however they return a specific word in the lower 5 bitsof 2002h, and registers 2000h and 2001h are flipped around. I'm fairlycertain that these are all the PPU's that exist. I have a good crosssection of games now.

This section is a tad bare right now... I'm still trying to figure outexactly how to flesh this out. This should be a good start, however.

If anyone is interested in the things nintendo did to make your lifedifficult as an arcade operator, here it is:

a) Different PPUs. There are 13 different PPU chips made that you can find on Vs. arcade boards.

b) Different controller pinouts. Some games came with new control panels you had to install with the game. This was pretty basic stuff and just remapped a few of the buttons.

c) Atari/Namco/Tengen came up with at least three different protection chips which map in the 5000-5FFFh area that the game checks. If the chip does not return the correct data, the game hangs or fails to start.

I really the "RAM size" bits are a bad idea. To know how to handle the RAM, you need to know the board's wiring anyway, except for the standard case of 8kB RAM at $6000. So the RAM size should be fully determined by the sub-mapper.

For instance, how would you handle contradictions between the sub-mapper and the RAM size bits? Having the same information in two different places in the file sounds like a bad idea to me. Which information source is the more trustworthy of the two?...

The one that was defined for the 2.0 version. The other just remains there for compatibility reasons. If old information contradicts new information, go with the newer one.

Actually, both of those fields are new.

One argument I have in favor of indicating both is that it allows reading the RAM sizes without having to look up the mapper+submapper - if you want to, say, filter your ROMs by the amount of save RAM they use, it would be rather annoying to have to look up all of the mappers and submappers every single time (not to mention requiring the program itself to be updated every time a new mapper/submapper is allocated).

Besides, there could potentially be places where separate submappers for different RAM sizes would not be optimal - the mapper logic might not differ substantially, or there might not be room for enough submappers to distinguish every variant.

You see it as redundant and a potential for contradictions. I see it as an additional useful way of presenting information and the ability to override values (in this case, I would say that the RAM counts override the submapper, since the submapper can distinguish far more than RAM sizes).

_________________Quietust, QMT Productions
P.S. If you don't get this note, let me know and I'll write you another.

If nothing else, please provide a version number somewhere in the spec. That is the one thing Marat failed to include that led to all the problems we have now. An emu MUST know which revision it is looking at. What happens if some special-case cartridge comes up where the info in this revision isn't sufficient? Extendibility requires a method to know whether or not the extension should be acknowledged by the emulator. Yes, the two bits in the eighth byte are good enough (in at least most cases, which is about all you can expect anymore) to distinguish this revision from the previous one, but how do we distinguish this revision from the next one (assuming there is)? I did not see anything in the spec that would allow for this possibility.

If INES originally had a version number, we wouldn't have to search the header for DiskDuede or anything before acknowledging the eighth byte - if it was version 0, the second flags byte would have been disregarded, while version 1 would have allowed the upper four bits to be used, and version 2 the lower 2 bits (VS and PC10). An emu who only knew version 0 would give a warning ("Image may require a feature I don't know about. Proceed anyawy?") if it saw a version 1 or 2 image. Problem solved. But alas, those days cannot be relived, but at least we can try to avoid a similar problem in the future (assuming this standard goes anywhere).

One more thing. If a game has battery-backed RAM and non-battery-backed RAM, how does an emulator know which segment should be loaded first (e.g. is it bank 0 or bank 1 that gets saved)?

Banamnos looks like thinking the exact opposite as I do.
Anyway, most people seems to have adopted the submapper idea, so I don't have to do anything agains it, but I think submappers are only needed on a few rare mappers. They aren't needed on standard cards, because the SRAM size bits tells us everything.

A few improvements need to be done, because I think dvdmth is right. Also I'm sad that my region idea wasn't taken in account.

_________________Life is complex: it has both real and imaginary components.

Imagine a file format consisting of the signature, a version number, then data that's version-specific. At some point, a new version is created that completely changes the format of the data after the version. For all intents and purposes, it's a new file format. Since iNES only has 16 bytes in the header, that's about the only approach you can take with a version number.

If iNES were a block-oriented format, you could just add your new information as a new block type that would be ignored by older programs. Presumably it would only add to the information about the file. With the 16 byte header of iNES, the only way to add new information without removing the old is to define more of the unused bytes.

In other words, I'm not imagining scenarios where a version number would help.

A related idea that I can imagine uses for are some mapper+submapper combinations making use of otherwise-unused bytes. This also relates to the 4-bit fields that specify various memory sizes. I think their value might be understood by rewinding a bit in the extension of iNES. Think of taking the current iNES and adding a new sub-mapper field. You'd find that many variants were simply specifying different memory sizes, and probably end up with lots of similar-but-subtly-different code in various mappers that decodes these fields.

Rather than do this, you have dedicated memory size fields to go along with the sub-mapper bits. This makes the meaning of each more consistent, leaving the sub-mapper bits for unique features of the main mapper number. The issue of memory size being insufficient because it could be wired different ways would be handled with a sub-mapper bit, since this is truely unique information.

The issue of potential inconsistent information is an important one to consider. In this case, the mapper itself defines what additional fields are even relevant. There's no way around this, as even the most basic options like "WRAM is battery-backed" and "H/V mirroring" don't apply to some boards. So the memory sizes are meaningful only in the context of a particular mapper. I'd think that one important guideline for consistency would be to avoid using the sub-mapper field if one of the memory size fields were sufficient.

Imagine a file format consisting of the signature, a version number, then data that's version-specific. At some point, a new version is created that completely changes the format of the data after the version. For all intents and purposes, it's a new file format. Since iNES only has 16 bytes in the header, that's about the only approach you can take with a version number.

<snip>

A related idea that I can imagine uses for are some mapper+submapper combinations making use of otherwise-unused bytes. This also relates to the 4-bit fields that specify various memory sizes. I think their value might be understood by rewinding a bit in the extension of iNES. Think of taking the current iNES and adding a new sub-mapper field. You'd find that many variants were simply specifying different memory sizes, and probably end up with lots of similar-but-subtly-different code in various mappers that decodes these fields.

Rather than do this, you have dedicated memory size fields to go along with the sub-mapper bits. This makes the meaning of each more consistent, leaving the sub-mapper bits for unique features of the main mapper number. The issue of memory size being insufficient because it could be wired different ways would be handled with a sub-mapper bit, since this is truely unique information.

The issue of potential inconsistent information is an important one to consider. In this case, the mapper itself defines what additional fields are even relevant. There's no way around this, as even the most basic options like "WRAM is battery-backed" and "H/V mirroring" don't apply to some boards. So the memory sizes are meaningful only in the context of a particular mapper. I'd think that one important guideline for consistency would be to avoid using the sub-mapper field if one of the memory size fields were sufficient.

Yeah, I should've stressed that before about the submappers... they are pretty much a "last resort". MMC5 is a good example of where the WRAM sizes are very useful- there are at least 4 kinds of MMC5 WRAM setups, while the MMC5 part itself does not change. Here are a few I know of:

This is the ideal problem the WRAM size fields were designed to solve.

As for the sub-mapper info, as Blargg explains, it's for hardware hookup differences that you may not be able to suss out with the information given in the header, such as the afore-mentioned mapper 16 and mapper 83 stuff.

If iNES were a block-oriented format, you could just add your new information as a new block type that would be ignored by older programs. Presumably it would only add to the information about the file. With the 16 byte header of iNES, the only way to add new information without removing the old is to define more of the unused bytes.

Or to define a footer that always occurs 16 + nPRGBanks*16384 + nCHRBanks*8192 bytes into the file. Make this footer block-oriented, and you have my iNIF suggestion.

And with this, I have started implementing it in the FPGA NES. I will have some more substantial submapper setups soon. I have "large ROM" support so far added, and I ripped out and re-did my loading routines to follow the spec. Once finished, I will have a GoodNES style tool that will fix all the broken ROM headers for all current ROMs, and full documentation on the submappers and such. Stay tuned.

Who is online

Users browsing this forum: No registered users and 7 guests

You cannot post new topics in this forumYou cannot reply to topics in this forumYou cannot edit your posts in this forumYou cannot delete your posts in this forumYou cannot post attachments in this forum