We’re almost done with our VFS implementation. The only thing left to discuss is the run-time functionality. The Pack and Unpack methods are used for offline tools. But in order to use our PAK file for run-time phase then we need to have some methods to open and only extract the necessary data we need. For loading our data, we’ll be loading it into the memory instead of saving it as a file. This not only saves time but also for security. It’ll be harder for hackers to steal our resources if we load our data in memory instead of an actual physical file.

The block of code below is our object initialization of the Open() method.

After acquiring the PAK info, this should be enough and we pop out of this function. But the problem with how we store our directory and file table is that its in a list! There is now way we can easily find the files that we want fast! To fix this problem, we’ll be creating a map storing our DirFileEntries using their path appended with DirFileEntry’s file name as a string ID. This way we can have about a N(log n) lookup performance. Instead of linearly searching the file in a list. After generating our file map we simply clean our resources and return.

There is a limitation on how we’ve set our Filemap. We’ll discuss this later at the end of this article.

Closing the opened PAK file is as simple as calling Close() which simply calls ReleaseResources() underneath.

FileSystem method – FindFile()

FindFile simply looks for the file in the map file and returns a handle to it. We never really discussed how this handle works earlier. So I would like to take the time to explain it here now. This method return OHFile which is a typedef of the Optional<> class. What makes this Optional so special is the way it handles return values and lets the user knows if the return value is valid or not. This avoids having unnecessary error return values and awkwardly placing the actual output in one of the parameter list. We can simple determine the return value if it is valid or not by calling Optional::IsValid(). Then we can access the value by simply prefixing ‘*’ on the optional value. You may also notice that the typedefed Optional<> class is containing a pointer

typedef Optional<PTR_T> OHFile;

PTR_T is simply typedefed as void *. The reason we want to do this is because when we find the fileEntry in the file map, we don’t want the user to actually mess around with it. So what we want to do is remove the interface from it. As if the return value is some kind of an ID. Here is the code below.

FileSystem::OHFile FileSystem::FindFile( const CHAR *pPath ) const

{

FileEntryMap::Iterator iter = m_pFileMap->Find( pPath );

if ( iter == m_pFileMap->IteratorEnd() )

return OptionalEmpty();

returnreinterpret_cast<PTR_T>((&(*(*iter).second)));

}

casting the found file entry to a PTR_T(void *)

We’ll see an example later when we finish discussing the remaining run-time methods.

FileSystem method – GetFileSize()

This method returns the size of the found file which accepts a filehandle returned by FindFile. The 64-bit version of this is simple called GetFileSize64(). The reason we have a 64 bit version return type is that it is possible that the file we’re looking for can be greater than 4GB large. In that case SIZE_T can’t return a value greater than that value.

This method can extract a file inside a PAK file and load it to the memory. Due to the slight complexity of how we’re handling the accepted OHFIle, we need to do some type casting to convert it into a DirFileEntry pointer. Once we have the DirFileEntry pointer, we can easily access where it is located in the PAK file then load it into the memory specified by pBuff. Take note that it needs to undergo the Decode process in order to reverse the filters applied to it (like zlib compression for example).

BOOL FileSystem::ReadFile( const OHFile *pOHFile, BYTE *pBuff )

{

if ( !IsOpen() && pOHFile->IsInvalid() )

return FALSE;

// Just use pointer to avoid additional processing by SmartPointer<>

FileReader reader;

MemoryWriter writer;

DirFileEntry *pFileEntry = ((DirFileEntry *)(**pOHFile));

m_pPulseFile->Seek64( pFileEntry->m_PAKData.m_diskStart );

reader.SetFileStream( m_pPulseFile );

reader.SetReadLimit( pFileEntry->m_PAKData.m_compressedSize );

writer.SetBuffer( pBuff );

Decode( pFileEntry, &reader, &writer );

return TRUE;

}

The End Of Our Implementation

If you have managed to read this far then congratulations! I seriously did not anticipate that it would reach this long (about 10,500+ words). But if you’ve been following along, then you now understand how a simple File System works! Noted that it still has a number of limitations but this already works for most of our needs and it wouldn’t take much time refactoring the code in order to further improve the system. I’ll be showing a quick example on how to use our File System then we’ll quickly talk about some things that we can do to further improve our system.

Using our VFS

To create a PAK file we create an instance of our FIleSystem then call Pack().

// NOTE: Callback that uses FILTER_TYPE_ZLIB_TEST compression/decompresson on all files

In this example we want to pack the entire contenst inside “C:\Program Files\Adobe” then save the PAK file in “C:\New Folder\” named as “TestPAK,pfs”. We’ve also passed in a function called SelectFiler and indicating that we’ll be using PulseFile::FILTER_TYPE_ZLIB_TEST filter.

The screenshot shows that we have managed to compress the entire adobe folder by 218%.

I’ll now show you how easy it is to unpack all of the contents of the pack file. If you’ve seen that darkened code below PFS_ACTION == 2 then that’s all you need to do. Calling Unpack() method specifying the path of the PAK file in the first parameter and the output directory in the second parameter.

You may have noticed a little discrepancy of the total size of the original and unpacked file. The reason for this is that there are some small system files inside the Adobe folder. Remember that we choose not to include the system and hidden files in our packing process.

And lastly, for loading a file to memory from a PAK file, you need to call FindFile() for searching for the file you are looking for. When it succeeds, it returns a valid Optional File Handle. Checking if the handle is valid is as easy as calling Optional<>::IsValid(). Once you found a valid handle, make sure you allocated enough memory space by calling GetFilzeSize, or GetFileSize64 for greater 4GB. Then finally, we call ReadFile() for reading the entire file to memory.

Before we say goodbye, I would like to discuss some few more things regarding our system. First of all, as you may have noticed, this system is not 100% complete! This is pretty much just a barebone File System structure. You can further enhance this by abstracting platform specific code like how we search files and directories which uses Win32 specific methods. Another thing to take note is that if you’ll be using this on a Unix platform(am i saying this right?), the separator for paths uses a forward slash’/’ instead of back-slash character. We can simply fix this by using a macro and identifying whether we’re under windows or unix platform. Another thing to take note is how we handle our bookkeeping of our files which is stored in a map. This is a good way for finding files you are looking for but very2x bad, if even possible, for querying the contents of a directory inside the PAK file. One way to fix this is to create a directory map instead that contains another map for the files in that directory.

—————————

If you have any questions, concerns, or pretty much just about anything you want to ask, feel free to email me.

I am also open to any feedbacks about this system as I will be further improving this. So definitely email me if you have a better idea for this implementation.