Pokémon Mystery Dungeon (WiiWare) data1_XXXX.bin and data2_XXXX.bin archive information
The files, data1 and data2 are archive binaries, the "XXXX" indicated in the file name, is the game code for the files, which can vary depending on the version (eg. data2_WPAJ.bin).
Since data1 and data2 archive binaries are compressed with the AT7 compression algorithm, the archives must be decompressed before any data can be extracted.
But here's how the decompressed data1 and data2 files are structured:
Part 1 - Pointer and file name table Part 2 - Actual data of all files
Part 1 - Pointer and file name table
This data contains all the pointers of all files, as well as their file names.
Each table row consists of:
Offset
Endianness
Type
0x0-3
Big Endian
Data location offset
0x4-7
Big Endian
File size
0x8-1B
Big Endian
Namespace data
Namespace data
This data contains the filename data, and all filenames can only be in ASCII format. Filenames can only be a maximum of 19 characters long, every filename always ends with a 00 value byte and must be in the namespace data in order for it to function properly. Anything after the 00 valued byte through to offset 0x1B will be what we'll refer to it as "junk text".
Junk text is the remains of what used to be a previous file that existed on that entry before it was overwritten by a different filename, either through renaming or removal of a file during development. So essentially it would be development leftovers. It's possible to create a program that would re-replicate the same type of junk data results though file namespace overwriting. For example, we have a file called adev_app_icon.tex, then we name it "app_icon.tex", it would look like this: "app_icon.tex tex". Alternatively if we renamed or removed "bg_event.sed", the file name in the table row below it called "bg.swd" would take priority and overwrite the text (and file offset data) on that table row and the namespace would look like "bg.swd t.sed", then if the file was renamed again to "b.swd", it would be "b.swd t.sed" which would look like "62 2E 73 77 64 00 00 74 2E 73 65 64 00 00 00 00 00 00 00 00" in hex. This means that an addition of a new file or renaming a file in the archive has the potential to create junk data, overwriting text in an existing entry while leaving some of it there.
Part 2 - Actual data
Each file data fits into this area as followed:
First - File data
Second - Line break filling
Line break filling
This data occurs between each file to show that its sepparate data, it can either be 16 or 32 bytes long, the FF value marks the end of the file data and start of the filling data, all the rest of the filling data is 00 value bytes, after that, the next file begins. The end of the entire data1 or data2 archive is marked with a FF value byte as well, but without the 00 values after it.
32-byte filling data:
FF 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
16-byte filling data:
FF 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Credits:
MegaMinerd - For most of the research into filling data

Pokémon Mystery Dungeon (WiiWare) data1_XXXX.bin and data2_XXXX.bin archive information
The files, data1 and data2 are archive binaries, the "XXXX" indicated in the file name, is the game code for the files, which can vary depending on the version (eg. data2_WPAJ.bin).
Since data1 and data2 archive binaries are compressed with the AT7 compression algorithm, the archives must be decompressed before any data can be extracted.
But here's how the decompressed data1 and data2 files are structured:
Part 1 - Pointer and file name table Part 2 - Actual data of all files
Part 1 - Pointer and file name table
This data contains all the pointers of all files, as well as their file names.
Each table row consists of:
Offset
Endianness
Type
0x0-3
Big Endian
Data location offset
0x4-7
Big Endian
File size
0x8-1B
Big Endian
Namespace data
Namespace data
This data contains the filename data, and all filenames can only be in ASCII format. Filenames can only be a maximum of 19 characters long, every filename always ends with a 00 value byte and must be in the namespace data in order for it to function properly. Anything after the 00 valued byte through to offset 0x1B will be what we'll refer to it as "junk text".
Junk text is the remains of what used to be a previous file that existed on that entry before it was overwritten by a different filename, either through renaming or removal of a file during development. So essentially it would be development leftovers. It's possible to create a program that would re-replicate the same type of junk data results though file namespace overwriting. For example, we have a file called adev_app_icon.tex, then we name it "app_icon.tex", it would look like this: "app_icon.tex tex". Alternatively if we renamed or removed "bg_event.sed", the file name in the table row below it called "bg.swd" would take priority and overwrite the text (and file offset data) on that table row and the namespace would look like "bg.swd t.sed", then if the file was renamed again to "b.swd", it would be "b.swd t.sed" which would look like "62 2E 73 77 64 00 00 74 2E 73 65 64 00 00 00 00 00 00 00 00" in hex. This means that an addition of a new file or renaming a file in the archive has the potential to create junk data, overwriting text in an existing entry while leaving some of it there.
Part 2 - Actual data
Each file data fits into this area as followed:
First - File data
Second - Line break filling
Line break filling
This data occurs between each file to show that its sepparate data, it can either be 16 or 32 bytes long, the FF value marks the end of the file data and start of the filling data, all the rest of the filling data is 00 value bytes, after that, the next file begins. The end of the entire data1 or data2 archive is marked with a FF value byte as well, but without the 00 values after it.
32-byte filling data:
FF 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
16-byte filling data:
FF 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Credits:
MegaMinerd - For most of the research into filling data

AT7 File Container Format
Documentation for...
Pokémon Mystery Dungeon: Keep Going! Blazing Adventure Squad
Pokémon Mystery Dungeon: Let's Go! Stormy Adventure Squad
Pokémon Mystery Dungeon: Go For It! Light Adventure Squad
Or by it's other names...
Pokémon Mystery Dungeon: Forward! Adventurers of Flame
Pokémon Mystery Dungeon: Let's Go! Adventurers of Storm
Pokémon Mystery Dungeon: Aspire! Adventurers of Light
All on WiiWare~
Hello everyone, I've actually been making some progress with the research into the AT7 File Container format, here's what I have currently documented:
As of now, I've done some huge research into the AT7 Container format
and I have made some progress (finally). The AT7 format is split into
multiple segments and sectors. Each segment has a 6-byte long header, both which
indicate it's ID and size of the segment. AT7P counts the header, where as
AT7X does not. Like AT4PX or PKDPX, the segment headers have control flags in
it's header, but it is only determined within the first AT7P segment header .AT7P segments
have only been examined with compressed data sizes of less than 49,158 bytes. It is unknown
if the AT7P segments can go higher than 49,158 bytes
AT7X segments have uncompressed data, but can only be 49,158 bytes long.
Now here's some examples of where you can find the AT7 Segments for data2_WPAJ.bin:
Address
Type
Segment Size
0x0 - 42F8
AT7P
42F9 bytes (Dec: 17,145 bytes)
0x42F9 - 87C1
AT7P
44C8 bytes (Dec: 17,608 bytes)
0x11589 - 1D58E
AT7X
C006 bytes (Dec: 49,158 bytes)
0x1D58F - 29594
AT7X
C006 bytes (Dec: 49,158 bytes)
0x29595 - 3559A
AT7X
C006 bytes (Dec: 49,158 bytes)
0x3559B - 415A0
AT7X
C006 bytes (Dec: 49,158 bytes)
..
0x14AAD78-14BFFF0
AT7E
15288 (Dec: 86,664 bytes)
(This would keep going on and on, can't post all of the offsets
since it would take too long, but I am posting about AT7E though)
Segments
AT7P segments can be any size! The size of the segment is always stated at offsets 0x4-5
in little endian. Where as AT7X is always at a fixed size and have uncompressed data,
therefore meaning that offsets 0x4-5 will only count the uncompressed data, but not the
header. The entire AT7 file container is split into sectors, containing 131,072 bytes
(20000h) each. The AT7E segment always indicates the end of the archived files and then
the free space comes after the AT7E header.
Compressed data works as follows: After the compressed size bytes, it begins with
each of the blocks. Each block starts with a flag byte. As it goes from MSB to LSB,
if the bit is set, it reads, takes the next byte from the input and moves it to the
output completely unchanged. If the bit is clear, it reads a 2-byte big-endian
control code. The top nybble is a repeated string length of 3. The remaining bits
are a 12-bit negative offset of the most recent occurrence of the string to repeat.
AT7P Segment Info
Offset
Name
Type
Endianness
Size
0x0-3
Text (AT7P)
-
Big Endian
4 bytes
0x4-5
AT7P Length*
uint16
Little Endian
2 bytes
0x6
Compressed data begins
-
-
-
* - These bytes count both the header and compressed data
As explained before, AT7X segments are a fixed size of 49,158 (C006) each, counting the
header as well. Unlike with AT7P, AT7X segments have uncompressed data, which means that
the data can be read without any need to decompress. Since AT7X can only hold data sizes
of 49,152 bytes, any data contained has to be exactly 49,152 bytes. This can often be
used for textures or certain images or any data that has no data that can be re-replicated
in any way, but must be exactly 49,152 bytes within parts of large files.
AT7X Segment Info**
Offset
Name
Type
Endianness
Size
0x0-3
Text (AT7X)
-
Big Endian
4 bytes
0x4-5
Data size***
uint16
Little Endian
2 bytes
0x6-C006
Uncompressed data
-
-
49,152 bytes (fixed size)
** - It always has a fixed length of 49,158 bytes (C006h) each
*** - The bytes are always 00 C0 (C000h), which means it is always 49,152 bytes long in decimal. Unlike AT7P, these bytes only count the compressed data, but not the header
The AT7E segment marks the end of the container and will fill any remaining bytes within a
sector with byte values of "45" (or "E" in ASCII).
AT7E Segment Info****
Offset
Name
Type
Endianness
Size
0x0-3
Text (AT7E)
-
Big Endian
4 bytes
0x4
Free space*****
-
-
-
**** - This segment always indicates the end of the AT7 container, before the free space
***** - These bytes can be any size, the entire AT7 container is made of blocks that are 131,072 bytes (20000h) each, and all free space bytes have a value of "45". The size of the free space will depend on how much space the last block has left over. For example, the last block of data2_WPAJ.bin only has AD7C (44,412 bytes) of data left in it 20000h - AD7Ch = 15284h (86,660 bytes), which means data2_WPAJ.bin only has 86,660 bytes of free space with each byte value of "45"
Sectors
Sectors are sections of the container that make up the file, like how sectors work on a Hard Disk Drive. Each sector is 131,072 bytes long (20000h). There can be multiple sectors within an AT7 container, for example, data2_WPAJ.bin contains 166 sectors (A6 in hexadecimal). But some contain as little as just 1 sector (such as data1_WPAJ.bin). If the last sector of the AT7 container has free space, the end of the data will be flagged with the AT7E segment, beginning the start of the free space.
Special thanks to:
MegaMinerd - For discovering and explaining about how the compressed data works and explaining that AT7X data isn't compressed

AT7 File Container Format
Documentation for...
Pokémon Mystery Dungeon: Keep Going! Blazing Adventure Squad
Pokémon Mystery Dungeon: Let's Go! Stormy Adventure Squad
Pokémon Mystery Dungeon: Go For It! Light Adventure Squad
Or by it's other names...
Pokémon Mystery Dungeon: Forward! Adventurers of Flame
Pokémon Mystery Dungeon: Let's Go! Adventurers of Storm
Pokémon Mystery Dungeon: Aspire! Adventurers of Light
All on WiiWare~
Hello everyone, I've actually been making some progress with the research into the AT7 File Container format, here's what I have currently documented:
As of now, I've done some huge research into the AT7 Container format
and I have made some progress (finally). The AT7 format is split into
multiple segments and sectors. Each segment has a 6-byte long header, both which
indicate it's ID and size of the segment. AT7P counts the header, where as
AT7X does not. Like AT4PX or PKDPX, the segment headers have control flags in
it's header, but it is only determined within the first AT7P segment header .AT7P segments
have only been examined with compressed data sizes of less than 49,158 bytes. It is unknown
if the AT7P segments can go higher than 49,158 bytes
AT7X segments have uncompressed data, but can only be 49,158 bytes long.
Now here's some examples of where you can find the AT7 Segments for data2_WPAJ.bin:
Address
Type
Segment Size
0x0 - 42F8
AT7P
42F9 bytes (Dec: 17,145 bytes)
0x42F9 - 87C1
AT7P
44C8 bytes (Dec: 17,608 bytes)
0x11589 - 1D58E
AT7X
C006 bytes (Dec: 49,158 bytes)
0x1D58F - 29594
AT7X
C006 bytes (Dec: 49,158 bytes)
0x29595 - 3559A
AT7X
C006 bytes (Dec: 49,158 bytes)
0x3559B - 415A0
AT7X
C006 bytes (Dec: 49,158 bytes)
..
0x14AAD78-14BFFF0
AT7E
15288 (Dec: 86,664 bytes)
(This would keep going on and on, can't post all of the offsets
since it would take too long, but I am posting about AT7E though)
Segments
AT7P segments can be any size! The size of the segment is always stated at offsets 0x4-5
in little endian. Where as AT7X is always at a fixed size and have uncompressed data,
therefore meaning that offsets 0x4-5 will only count the uncompressed data, but not the
header. The entire AT7 file container is split into sectors, containing 131,072 bytes
(20000h) each. The AT7E segment always indicates the end of the archived files and then
the free space comes after the AT7E header.
Compressed data works as follows: After the compressed size bytes, it begins with
each of the blocks. Each block starts with a flag byte. As it goes from MSB to LSB,
if the bit is set, it reads, takes the next byte from the input and moves it to the
output completely unchanged. If the bit is clear, it reads a 2-byte big-endian
control code. The top nybble is a repeated string length of 3. The remaining bits
are a 12-bit negative offset of the most recent occurrence of the string to repeat.
AT7P Segment Info
Offset
Name
Type
Endianness
Size
0x0-3
Text (AT7P)
-
Big Endian
4 bytes
0x4-5
AT7P Length*
uint16
Little Endian
2 bytes
0x6
Compressed data begins
-
-
-
* - These bytes count both the header and compressed data
As explained before, AT7X segments are a fixed size of 49,158 (C006) each, counting the
header as well. Unlike with AT7P, AT7X segments have uncompressed data, which means that
the data can be read without any need to decompress. Since AT7X can only hold data sizes
of 49,152 bytes, any data contained has to be exactly 49,152 bytes. This can often be
used for textures or certain images or any data that has no data that can be re-replicated
in any way, but must be exactly 49,152 bytes within parts of large files.
AT7X Segment Info**
Offset
Name
Type
Endianness
Size
0x0-3
Text (AT7X)
-
Big Endian
4 bytes
0x4-5
Data size***
uint16
Little Endian
2 bytes
0x6-C006
Uncompressed data
-
-
49,152 bytes (fixed size)
** - It always has a fixed length of 49,158 bytes (C006h) each
*** - The bytes are always 00 C0 (C000h), which means it is always 49,152 bytes long in decimal. Unlike AT7P, these bytes only count the compressed data, but not the header
The AT7E segment marks the end of the container and will fill any remaining bytes within a
sector with byte values of "45" (or "E" in ASCII).
AT7E Segment Info****
Offset
Name
Type
Endianness
Size
0x0-3
Text (AT7E)
-
Big Endian
4 bytes
0x4
Free space*****
-
-
-
**** - This segment always indicates the end of the AT7 container, before the free space
***** - These bytes can be any size, the entire AT7 container is made of blocks that are 131,072 bytes (20000h) each, and all free space bytes have a value of "45". The size of the free space will depend on how much space the last block has left over. For example, the last block of data2_WPAJ.bin only has AD7C (44,412 bytes) of data left in it 20000h - AD7Ch = 15284h (86,660 bytes), which means data2_WPAJ.bin only has 86,660 bytes of free space with each byte value of "45"
Sectors
Sectors are sections of the container that make up the file, like how sectors work on a Hard Disk Drive. Each sector is 131,072 bytes long (20000h). There can be multiple sectors within an AT7 container, for example, data2_WPAJ.bin contains 166 sectors (A6 in hexadecimal). But some contain as little as just 1 sector (such as data1_WPAJ.bin). If the last sector of the AT7 container has free space, the end of the data will be flagged with the AT7E segment, beginning the start of the free space.
Special thanks to:
MegaMinerd - For discovering and explaining about how the compressed data works and explaining that AT7X data isn't compressed

While we have been doing a lot of research on Explorers, but there's also another game I've been interested in researching lately.
Since Pokémon Mystery Dungeon: Forward! Adventurers of Flame, Pokémon Mystery Dungeon: Let's Go! Adventurers of Storm and Pokémon Mystery Dungeon: Aspire! Adventurers of Light never got released outside of Japan. Little has been documented about these games, but they use very similar file formats to the Explorers ones. So the Adventures ones also could be included in PMD2 as well, since they're part of the same generation, but instead released as WiiWare on the Wii.
One particular file format that drew my attention, was the AT7P file container format. I haven't done much research on it, but it is an archive format that also uses compression (possibly lz77 or yaz0).
Unless we're able to extract and repack this file format, there won't be much hope of a translation project occurring anytime in the future. Which is why I'm interested in researching this file format. So far from what I've found apart from it's header ID "AT7P" at 0x0-2, I've found that free space within this file is indicated by every byte as "45". It could be related to some of the file container formats in the Explorers (such as AT4PX and AT4PN). I'll need to do some research, but if anyone else wants to, you'll need a program called ShowMiiWads in order to unpack the Wii .app files. There's two of these AT7P files contained in the app, they're under the names "data1_WPAJ.bin" and "data2_WPAJ.bin"
In the compressed data, I can see that data1_WPAJ.bin contains only a data.ini with some version info stuff inside data.ini. But it seems like data2_WPAJ.bin contains all the game data in it, only thing left to do, is to have a solution to unpack and repack the files. Just from looking at it, it seems to contain many SIR0 files as well.

Yep, exporting and importing still works.
But there's also some images that are not rendered correctly in Sky Editor. I'm pretty sure it is their native image format. But just the game's engine renders it differently.

I can understand that. I too have been playing so much of BotW as well. And I've also noticed that CTE/IMG functionality is broken in GTI. Using SkyEditor.ROMEditor.MysteryDungeon.PSMD.CteImage variable results in the no available UI message. While it kinda worked in legacy builds (with some images loaded incorrectly), I may need to do some research into the CTE image format. While it may display incorrectly, there's a chance that it may be in some compressed format, or it may rely on the game's engine to move the pixels around.

Ah, thank you. I'm now using the latest version, and it seems to me that I can't seem to see anything where I can sort it into the order of the file displaying from top to bottom. Usually having something like that usually makes it more easier for me to look through the files and see which ones are in the Adventure Hints category and so fourth.
And what I meant by having a number column, was actually to do with how the file is read from top to bottom, so it can be used alongside ID and Entry columns as well.

Alrighty, I've done some research into the SIR0 format on common.bin and 54.sir0, and I've noticed that the text in files, as well as it's scripts, are displayed in the right order when viewed from a Hex Editor, but when viewing from Sky Editor (a program that hasn't had any changes to it's code in over a year now), it displays any text entries in an incorrect order. Normally, it should also have the number of each entry from the start of the file, to the end of the file. But this is not the case with Sky Editor, it doesn't have a number column for each entry.
And I've noticed an oddity, some of the IDs in the list are displayed with a negative value. Are they meant to display a negative value? If so, then the ID might not be the problem.
Either way, text editing in both GTI and SMD are out of order, and something needs to be fixed with ROMEditor.FileFormats.MessageBin. If you'd like, I could dig into Sky Editor's code and see what seems to be wrong with the ROMEditor.FileFormats.MessageBin variable.

I am currently in the process of datamining Gates to Infinity. However, I've come across a file type called .img, which displays 2D images, such as text, textboxes and main menu artwork. Since I have no knowledge of programming, I can't create a program to convert the file type. But I have read up on Megadrifter's post on how the file format operates, unfortunately I'd need programming knowledge if I was to create a program to convert it.

Welcome back! It's been a long time since I've last come here and posted. I never would've expected the forums to be compromised by a malware attack. The worst thing about malware attacks... is that it encrypts files, rendering them completely unusable. I honestly hope nothing like this ever happens again. It would be horrible to lose more data. Dx
Anyways, I'm glad to see the forum up and running again. ^^

All it needed was a bit of scaling down the logo to be the exact size as that of the Japanese logo... and that's was it. As you can see, the original official English Pokémon Black/White Version had the logo pretty small, it had to fit in the 256x128 pixels margin. The image had to include both the checkered pattern as well as the "Pokémon Black Version" logo, as they both had to fit in the 256x128 pixels margin. Same thing applies for the Pokémon Black Version 2 logo, there is no way that even developers working for Nintendo themselves would be able to bypass those margins and go between them.
'Cause here's the thing... the NCGR files can only count higher in this pattern: 8, 16, 32, 64, 128, 256, 512, 1024 ect. It's just like the sizes in the .nds files themselves.