Let me start by saying this is a pretty long document. Sorry. If you bear with me, hopefully you'll find something useful in it all.

Over the last several days, I've discovered what all reversers eventually do: that lack of knowledge about the Import Table and Import Address Table (IAT) will severely hinder your efforts to learn about modern executable packers/encryptors. My increasing frustration finally gave way to me to sitting down with Firefox, opening up Iczelion's PE tutorials, the official PE/COFF standard documentation, and several other miscellaneous references and tutorials simultaneously along with a hex editor, LordPE, and ollydbg. After quite a few hours of flipping back and forth between all the tutorials and examining the structures in various real-world PEs, packed PEs, and raw dumped unpacked PEs, I think I finally got it to all *click* in my brain.

What I've learned through all of that, besides knowledge of these structures, is that there is a great deal of confusion surrounding them. The PE/COFF specification itself is vague, confusing, and the nomenclature for the PE structures is often misleading. For example, the ORIGINAL_FIRST_THUNK member is not a thunk - it's a pointer to an array, whose members are ordinals or pointers to function descriptions, neither of which are thunks, either! In addition, there's a general lack of knowledge and familiarity with the subject, while those who do understand it tend to see it through the microscope of a debugger or hex editor. The result is lot of unintentionaly misinformation and confusion for those who want to gain knowledge on the subject.

The purpose of this little paper is not be an end-all-be-all reference for the PE format, or even for the Import Table and IAT. For that, there is always the PE/COFF specification and Iczelion's fine tutorials, among other things. Instead, this is a general view of the structures which is written from the vantage of a novice who would like to clear things up and "set the record straight", so to speak. It's intended to give you the broader view of things so that the detailed specifications and explanations will make some damn sense! :P

First a word on thunks, and why we have an IAT to begin with. Because each process is contained in its own little virtual address space, and because the OS is responsible for loading a DLL into that space, a program cannot know what base virtual address a DLL is going to be loaded at when the program is compiled. Furthermore, the may be loaded at a different address every time the program is run. To fix this problem, the program doesn't call DLL functions directly. Instead, it calls the address pointed to by a known address. In assembly, one way of doing this is something like:

<program code>
CALL DWORD PTR DS:[00050000]
<more program code>

where 00050000 is a local address in the module. Ultimately address 00050000 will contain the address of the entry point for the function which we are trying to call. This mechanism is called a thunk. We put all of these "proxy" addresses together into a thunk table when we compile the program so that our code never makes a direct call to an extramodular address. We provide the operating system with a list of all the functions we want to import, and where in the table we need their addresses to be written so that our code will wind up calling the right location at runtime. The IAT is the thunk table which the loader builds for us.

As I mentioned in the introduction, this document is not a full PE tutorial. I will be skipping the unrelated structures entirely; this is just a schematic view, essentially. Below is the general layout of a PE file:

PE File

DOS Header

DOS Stub

PE Header

Section Table

Sections

The DOS header and stub are just there so that DOS can berate you for trying running a PE file without Windows :P. The PE loader skips these if it finds a PE header in the file. On an NT/2k/XP system, these really don't serve any purpose since the DOS VM is PE-aware anyway. The section table describes the name of each section, their offset and size in the file, and their relative virtual offset and size in memory as loaded by the PE loader.

Of interest to us is the PE header. The PE Header itself is made up of several parts:

PE Header

Signature

File Header

Optional Header

The signature is simply the ASCII string "PE" terminated by two zero bytes (from a hex editor you will see "50 45 00 00"). I won't get into the details of the File Header here except to mention that it specifies the length of the Optional Header, and the number of sections in the PE file (this tells us how many entries are in the section table).

The Optional Header is one of those misnamed elements. There's really nothing optional about it since it contains the address of the PE's entry point and the imagebase, among other critical things:

Optional Header

Address of Entry Point

Imagebase

Section Alignment

File Alignment

Major Susbsystem Ver.

Minor Susbsystem Ver.

Size of Image

Size of Headers

Data Directory

For the purpose of this document, we are unconcerned with all the items here except for the last one, the Data Directory, an array of 16 IMAGE_DATA_DIRECTORY structures:

Data Directory

Index

Structure

0

Export Table

1

Import Table

2

Resource Table

3

Exception Table

4

Certificate Table

5

Base Relocation Table

6

Debug

7

Architecture-specific

8

Global Pointer

9

Thread Local Storage Table

10

Load Config Table

11

Bound Imports

12

Import Address Table

13

Delay Import Descriptor

14

COM+ Runtime Header

15

Reserved (unknown)

The Data Directory is basically a table of contents for all the special structures in the file which the operating system and loader will use to do their magic. Each IMAGE_DATA_DIRECTORY item in this array consists of nothing more than a 32-bit RVA (pointer) to the special structure and a 32-bit size for the structure. Again, only the second and thirteenth items (index 1 and 12) are going to be discussed here.

So in brief, just keep this chain in mind:

The PE file contains the PE Header

The PE Header contains the Optional Header

The Optional Header contains the Data Directory

The Data Directory points to the Import Table and Import Address Table

For the time being, we're going to forget about the IAT pointed to by element 12 of the Data Directory. For now, it suffices to say that the IAT is derived from the Import Table by the loader at runtime, before the first instruction of the program is executed. It's also worth pointing out that the Import Table is frequently refered to as the Import Descriptor Table (IDT). This is mainly to help differentiate between the Import Table and the Import Address Table. Although these two structures are totally different things, they're related to one another, and the similarity in their names can be confusing. For the sake of brevity, I will sometimes refer to the Import Table as the IDT from here on out.

To make a long story short, the purpose of the IDT is to list each DLL that the executable will need, and each function in each of those DLLs that the executable is importing. The descriptions are given by name or by ordinal, depending on the requirements of the DLL. Locating all of these functions and filling in the IAT with their addresses is a multi-step process, so the IDT itself doesn't contain one huge list of all of the functions. Instead, it breaks things down into separate DLLs.

The IDT is an array of IMAGE_IMPORT_DESCRIPTOR structures, one for each DLL that the executable imports from. The end of the IDT is marked by an IMAGE_IMPORT_DESCRIPTOR filled with zeros (just like null-terminating a string, only bigger :P).

Import (Descriptor) Table

IMAGE_IMPORT_DESCRIPTOR for 1st DLL

IMAGE_IMPORT_DESCRIPTOR for 2nd DLL

IMAGE_IMPORT_DESCRIPTOR for 3rd DLL

...

IMAGE_IMPORT_DESCRIPTOR for Nth DLL

IMAGE_IMPORT_DESCRIPTOR containing zeros

Don't be intimidated by the fancy name - the IMAGE_IMPORT_DESCRIPTOR is little more than a pointer and a name. Each one is 20 bytes long and has the following structure:

IMAGE_IMPORT_DESCRIPTOR

Offset

Size

Description

0

4

Import Lookup Table (OriginalFirstThunk)

4

4

Time/Date Stamp

8

4

Forwarder Chain

12

4

Name

16

4

Import Address Table (FirstThunk)

Alright, here's all you need to know. First of all, fowarder chains are an advanced topic beyond the scope of this document. Under normal circumstances, we don't really care what this value is. The time/date stamp is largely irrelevant; usually the value of this dword in the file is either 00000000h or FFFFFFFFh, but it doesn't really matter. When the file is loaded, the loader will fill in this value, overwriting whatever was there. The interesting items are Name, Import Lookup Table, and Import Address Table.

The Name member is a 32-bit RVA pointer to an ASCII string terminated by a zero which gives the name of the DLL to import from. The Import Lookup Table is the description given in Revision 6.0 of Microsoft's PE/COFF specification, but this is not the name used by any assembler, compiler, or PE editor I've ever seen. Instead, this is almost universally called OriginalFirstThunk. Along the same lines, the Import Address Table member is usually called FirstThunk. Both of these names are bad misnomers since neither of these objects is a thunk, so it's actually kind of helpful to think of them by the friendly description instead. In a little bit, you'll see why these are called FirstThunk and OriginalFirstThunk.

We'll talk about the FirstThunk (IAT) member momentarily. For now, let's look at the OriginalFirstThunk member. This is just an 32-bit RVA pointer which points to our first IMAGE_THUNK_DATA structure. Each function we import will have its own IMAGE_THUNK_DATA structure. The last IMAGE_THUNK_DATA structure is filled with zeros to tell us when we've reached the end of the imports for this DLL.

The content of an IMAGE_THUNK_DATA depends on whether we're importing by name or by ordinal, but in both cases it is 32 bits long. IMAGE_THUNK_DATA structures don't actually contain thunks, but rather information that the loader uses to build thunks, so it's a bad choice of name. That said, let's look at the case of an import by name.

When a function is imported by name, the IMAGE_THUNK_DATA member will have a 0 in its most significant bit. This is what tells the loader how to interpert the rest of the IMAGE_THUNK_DATA structure. The remaining 31 bits are an RVA pointer to an IMAGE_IMPORT_BY_NAME structure. Although this pointer is only 31 bits long, the fact that the most significant bit is always zero in these circumstances lets us use the entire IMAGE_THUNK_DATA structure as a 32-bit pointer without having to do any masking operations. We need only check the highest order bit first to make sure that we're importing by name.

The IMAGE_IMPORT_BY_NAME structure is only a structure in a loose sense of the word. The first two bytes are always a hint value, which the loader can use to look up the function in the DLL's export table more quickly. However this value can be (and often is) zero, in which case the loader ignores it. It's just an optimization thing, and isn't necessary. Following this two byte hint is an ASCII string naming the function to be imported. The string is terminated by at least one zero. I say "at least" because the entire IMAGE_IMPORT_BY_NAME structure must be padded with an additional zero if necessary to make the structure end at an even byte boundary. Presumably, this is necessary because the loader performs some operation on the IMAGE_IMPORT_BY_NAME structures which requires proper data alignment. Data misalignment would probably end in disaster, so keep that in mind if you're trying to build these by hand.

The other way to import a function is by ordinal. An ordinal is just a ID number for the function. All of the export functions for a DLL that exports by ordinal are assigned a number from a contiguous numberspace in the DLL's export table. Importing by ordinal is basically just saying, "hey, I need function #XYZ from the DLL!" It's up to the DLL's creator to provide the programmer and their compiler with a definition of what ordinal corresponds to what function.

When we import by ordinal, the IMAGE_THUNK_DATA structure is again 32 bytes long. However, the most significant bit will be 1, which is how we differentiate between an import by name or an import by ordinal. The remaining 31 bits are the ordinal number we wish to import. The will use this value directly to look up the function in the DLL's export function, so unlike importing by name, there's no need for any further structures.

Yes, you can import some functions by name, and some functions by ordinal within the same DLL. In this case, some of the IMAGE_THUNK_DATA structures pointed to by OriginalFirstThunk will contain pointers to IMAGE_IMPORT_BY_NAME structures, and some of them will just contain ordinal numbers instead. Since IMAGE_THUNK_DATA is 32 bits long either way, this doesn't cause any problem. The loader can look at the most significant bit of each member in the array and figure out which it is.

Ok, I promised I'd talk about this after we'd covered the OriginalFirstThunk member. FirstThunk member is a 32-bit RVA pointing to points to another array of 32-bit values, equal in size to the array which the corresponding OriginalFirstThunk member points to. In the PE file (i.e. *before* the file is loaded into memory as an image), this array contains an exact duplicate of the information in OriginalFirstThunk's array. This is only a temporary arrangement, however.

When the loader arrives at the IMAGE_IMPORT_DESCRIPTOR structure for a DLL, it looks for the OriginalFirstThunk member. It follows this pointer to the array of IMAGE_THUNK_DATA structures. It uses the ordinal stored there, or the IMAGE_IMPORT_BY_NAME pointed to by it to look up the virtual address of that function's entry point, as seen from the process's memory space. At the same time, it walks through the duplicate array pointed to by the FirstThunk member. Whenever it looks up the address of a function using the OriginalFirstThunk array, it writes the address to the corresponding member over the value of the corresponding element in the FirstThunk array. This, at last, is the Import Address Table. Well, at least the portion for that DLL -- the loader has to repeat the process for each DLL listed in the Import Descriptor Table.

When the loader finishes, all of the data we used to find the entry point address for each imported function is preserved, stored in the OriginalFirstThunk array should we ever feel like looking it up again. Our program won't use any of that information, but instead will use the values filled into the FirstThunk array as a thunk table.

If we rewind our thoughts a bit and go back to the Data Directory, you'll remember that it also contained a pointer (and size) for the IAT. The arrays collectively pointed to by the FirstThunk member of each IMAGE_IMPORT_DESCRIPTOR referenced in the IDT should more or less occupy a contiguous block of memory. The IAT entry in the Data Directory should point to the first byte of this block, and should be large enough to cover all of the FirstThunk arrays....in principle -- read the next section.

Since this was a lot of information to digest, here is basic roadmap of sorts of the import structures:

I've tried to depict what goes on with the IAT during the load process. Initially, before the loader gets ahold of them, the FirstThunk arrays in the IAT contain the same information as the OriginalFirstThunk arrays, pointing to the same IMAGE_IMPORT_BY_NAME structure, or containing the same ordinal number. After the loader processes the OriginalFirstThunk arrays, the FirstThunk arrays contain true thunks to the function entry points, so the little arrows to the IMAGE_IMPORT_BY_NAME structures would go away :P

This is a section which covers some of the things that you might not think about when dealing with imports. Hopefully this will help you avoid making some common mistakes or getting confused by some tricky aspects of this whole convoluted system.

All RVAs, hint values, etc. are stored in little-endian format. If you're familiar with Intel architecture, you already knew this, but it's worth pointing out. It means that if you're poking around with a hex editor or debugger, an RVA like 000A1358 will be stored as: 58 13 0A 00.

RVAs are not virtual addresses, they are relative virtual addresses. You must add the imagebase value to get the actual virtual address.

Converting an RVA to a file offset is not as trivial as you think. You must look at the section table and determine which section the RVA belongs to. Then calculate the difference between the raw (file) offset of the section from the starting RVA for the section. If the section's RVA is greater than the file offset of the section, this value must be subtracted from your RVA to convert it to a file offset. If the sections raw offset is greater than its starting RVA, you must add the difference to your RVA to convert it to a file offset.

The order of the DLLs' OriginalFirstThunk and FirstThunk arrays in memory does not necessarily follow the order of the DLLs in the IDT. That is, all of the IMAGE_THUNK_DATA members for a single DLL will be together as a contiguous block in memory, but the block for the first DLL in the IDT could come after the block for the last DLL in the IDT in memory.

Likewise, the order of the IMAGE_IMPORT_BY_NAME structures pointed to by the IMAGE_THUNK_DATA structures is not guaranteed to be in any particular order in memory

The IAT entry in the Data Directory *ought*, at the very least, to define a block of memory large enough and in the correct position to contain all of the FirstThunk arrays for all of the DLLs in the IDT. However, I have seen executables for which the starting address and size for the IAT listed in the IDT were both zero, and the PE still loads and runs normally. Thus, you should not count on the IAT entry in the Data Directory having any kind of sane values. If you really want to know where the IAT is, walk the entire chain of import structures and figure out what its bounds are.

Importing functions by name is almost always prefered over importing by ordinal. However, there are some DLLs which only import by ordinal, in which case you have no choice.

Well, now that we've covered how the IAT is created, it's worth examing how we put it to practice. In the introduction I brought up one method of making a call to a function referenced in the IAT. However, this is only a general approach. There are a two methods which are commonly employed in executables. First, let's assume we have an IAT that looks like this:

This jump table is a thunk table, of sorts, but it is not the same thing as the IAT. This sort of table is compiler and linker specific, and may not be present at all in a program. If it is present, it may be one big jump table, or there may be several jump tables scattered throughout the program. A program that uses a jump table may also make the first type of call instead at some places in the code.

At last we come around to what original started me down the path of import madness :P. There are several things that a packer can do that will screw up the import mechanisms so that you can't just dump the program and run it.

First of all, nearly all packers -- even "innocent" packers like UPX -- wind up destroying the IDT. The target executable is packed at the time the loader is run, so the loader can't very well read the target's IDT and fill in the IAT. Instead, the packer has to fill in the IAT for the program before jumping to the target's entry point. It could do this by exactly duplicating the function of the loader, and thus leave the IDT alone. In this case, all you would have to do is make sure that the Data Directory in your dump has a valid pointer to the IDT, and away you go.

Unfortunately, that's not the way things are normally done. Even if you aren't trying to protect the target exe, it doesn't make a lot of sense to keep the IDT around because it simply wastes space. There are more efficient ways to store import data, so the IDT is usually destroyed. However, for most simple packers, the IAT is still intact and valid. This structure has to be filled in correctly by the packer so that the target program's intermodular calls will work.

If we arrive at the original entry point of a program, and it hase a valid IAT, we can dump the file and use an import rebuilder like Imprec to rebuild the IDT using the IAT. To do this, the import rebuilder must attach to our still running process from which we made the dump. Once it has attached, it can discover where the DLLs were loaded into memory at the time the dump was made. It can then use the export table in the DLLs as a sort of reverse lookup to get the functions' names from their address in the target's IAT. Using this information it can completely rebuild the IDT.

Completely destroying the IAT is usually not a practical option for the packer, because it would have to replace every call in the program with some other mechanism. If the target program is self-modifying, or contains calls like: CALL EAX, the packer can never be entirely sure that it has found all of the calls into the IAT and fixed them.

However, many packers modify some or all of the addresses in the IAT to point into their own code. Within their code, they will resolve where the call needs to go and make the call as discreetly as possible. There are lots of different methods that can be employed here, depending on the packer. This is why when you run Imprec, for example, the first thing you do after finding the IAT is to fix invalid thunks. The invalid thunks are usually thunks that have been modified to jump through packer's code instead of directly to their original destination. You'll have to either trace through these yourself and fix the thunk to point to its original destination, or you'll have to have a plugin tracer that can. Only after the IAT is completely valid can the import reconstructor rebuild the IDT correctly.

There are a few other tricks employed by packers that are designed to circumvent the IAT entirely. They may go through the program and modify every call of the style: CALL DWORD PTR DS:[XXXXXXXX] so that those call directly into the packer's code, bypassing the IAT altogether. Or they may modify those JMP DWORD PTR DS:[XXXXXXXX] tables that I mentioned earlier. Either way, it accomplishes the same thing. Some of the standard import reconstruction tools have the ability to scan for different CALL types and try to fix them, but for newer protection methods, this may not work. This is where knowledge of the import mechanisms really comes in handy, as you'll have to write your own import repairing code.