Besides being the first Micosoft OS (Operating System) to feature an integrated Windows NT and Windows 98 kernel (the core program code of an OS), Windows XP Home Edition is also the first time home users have had the option to use the NTFS (New Technology File System).

The File System provides a foundation for storing data on a hard drive. Understanding the strengths and weaknesses of each File System can help you in choosing which one is best for you.

What Is A File System?

There is more than one way to store information on your hard drive. In addition to NTFS, XP supports the FAT-16 (16-bit File Allocation Table) and the FAT-32 (32-bit File Allocation Table) file systems. Both versions of FAT are simpler and more direct than NTFS but not as reliable or secure.

Due to it's reliability and security, NTFS has long been the file system of choice in business environments. With NTFS, system administrators can exercise some control over what users do on a workstation. And when problems occur, NTFS is better able to recover without losing data.

FAT has traditionally been the file system of choice for personal or home computers. Yet with the inclusion of NTFS support in Windows XP, significant numbers of home users are making the switch to enjoy the extra security and reliability NTFS provides.

File System Compatibility

File System compatibility becomes important if you plan to install more than on OS. In order to install more than one OS, you'll need to partition your hard drive. Windows treats each logical partition as a separate drive and each can contain its own File System.

In general, systems that can support NTFS also support FAT:
Windows NT
Windows 2000
Windows XP Home and Pro

This means Windows ME installed on a FAT partition can't access data stored in an NTFS partition. XP, on the other hand, can read and write data from both NTFS partitions and FAT partitions.

Note that WinNT does have some compatibility problems with newer versions of both FAT and NTFS. Although WinNT supports the older FAT16 file systems, it does not support FAT32. WinNT also has NTFS problems. Unless you have Service Pack 4 installed, you won't be able to read new NTFS 5.0 partitions that Windows 2000 and Windows XP use.

Understanding A File System

Before we discuss FAT and NTFS in detail, let's cover a few basics for those of us who are less-informed. Both types of file systems divide data into units called clusters. Each cluster can only hold one file, yet one file may span several clusters; but any empty space left in the last cluster remains vacant.

Smaller cluster sizes are more efficient from a storage perspective (they make for less leftover space), but they're less efficient in terms of performance. Larger clusters can hold more data, which means less work for the PC and slightly better performance. Cluster size also plays a role in deciding how large a hard drive a system can support. Overall, cluster size increases with available hard drive space.

On a newly installed system, related clusters are stored close together. As time passes, however, these clusters can become fragmented and find themselves scattered all over the hard drive. Most versions of Windows include Disk Defragmenter, a utility that examines data in each cluster and reorganizes it so related clusters are placed closer together on the hard drive. Fragmentation is a problem for both FAT and NTFS partitions.

---------------Cluster Chart---------------

Smaller clusters can squeeze more data onto a drive, but larger clusters provide better performance. Generally, 4KB clusters provide a nice balance between performance and storage efficiency. Below are the default cluster sizes for the FAT16, FAT32, and NTFS under Windows XP.

The most basic Windows file system is known as FAT. It gets its name from the file allocation table stored near the beginning of the drive. The FAT keeps track of every file saved on its drive or partition. When you save data or a file to the hard drive, the system consults the FAT to find empty clusters. After it saves the information, it modifies the FAT to reflect the name of the file saved and the clusters it saved the file to. When opening the document, the system searches through the FAT for the file name, finds the associated clusters where the file is stored, and reads the information contained in those clusters.

As I stated earlier, there are two primary varieties of FAT. FAT16, or 16-bit FAT, has been around for many years. Then FAT32, or 32-bit FAT replaced FAT16.

FAT16. FAT16 is a 16-bit file system because it identifies clusters that are 16 bits (or digits) long. There are quite a few possible combinations with 16 slots to work with, but there is a limit. Because FAT cluster sizes vary depending on how much drive space the system needs to catalog with its range of available names, and because FAT16 can't support as many clusters as FAT32, it needs larger clusters to cover the same amount of space. To format an entire 1BG partition, as example, FAT16 clusters must expand to roughly 32KB, which is too large for efficient storage.

XP supports FAT16 clusters up to 64KB in size. Given that FAT16 can only support 65,536 clusters, you can't use FAT16 on a partition larger than 4GB. Of course, 64KB clusters are extremely inefficient. About the only thing using FAT16 today are floppy drives and their 1.44MB disks.

FAT32. FAT32 made it's first appearance in Windows 95 OSR2 (Original equipment manufacturer Service Release 2; a version of Windows 95 made available exclusively to manufacturers and never sold at retail). Windows 98 was the first retail version of Windows to include support for FAT32.

Because FAT32 is a 32-bit file system, it can address more clusters than FAT16. As a result, FAT32 can use smaller clusters for better storage efficiency. It can also support larger partitions, up to 8TB (terabytes) in size.

FAT32 was made compatible (as much as possible) with FAT16, but some changes were made behind the scenes. As a result, some software, such as older drive utilities, will not work on hard drives using FAT32 file systems. Hard drive compression utilities, such as DriveSpace and DriveSpace 3 are incompatible with FAT32.

NTFS

For several years now NTFS has been well-known for its security and reliability. Windows XP Home Edition marks the first time home users get to use this more robust file system. However, there's more to NTFS than just laying out data on a hard drive.

Master File Table

If NTFS had anything in common with FAT, it would be the MFT (Master File Table). The MFT is very close to the file allocation table in FAT, only much more complicated. The MFT stores file attributes for every file stored on the NTFS partition.

File attributes describe everything there is to know about a file. In NTFS, even the data contained in a file is a file attribute. Additional file attributes include the file's name, location, and security information. When possible, NTFS stores all the file attributes, including the file's data, in the MFT. Often there is not enough room to accommodate all the attributes in the MFT so the data attribute will be moved outside the MFT, and a pointer will direct the system to all the clusters containing the data attribute of the file. The system can then retieve the data.

Metadata

The MFT is known as a metadata file. Metadata is essentially data about data. NTFS uses metadata files to manage data on the partition . Some of the more important metadata files include the MFT Mirror, Log File, Cluster Allocation Bitmap, Bad Cluster File, and Quota Table. It's pretty easy to figure out that the MFT Mirror is just a backup copy of the MFT. NTFS stores the MFT at the beginning of a hard drive's platter, and places the MFT Mirror in the middle of the platter. If the original MFT becomes damaged, the MFT Mirror will provide the necessary information to recover data on the system.

The Log File, also known as the Change Log Journal, records changes made to the file system. The Log File only records actions taken, not data which is modified. In other words, it can tell you when Windows writes a document to the file system but not what data the document contained. The Log File is useful to Anti-Virus programs, Backup utilities, and other applications which have an interest in knowing when changes occur to the file system.

The Cluster Allocation Bitmap is a map of the partition. The system uses the information contained in the Cluster Allocation Bitmap to locate available clusters to write new data to. The Bad Cluster File is similar, but it marks sections of the hard drive that have gone bad; and the system will not store data in these marked clusters.

The Quota Table is a feature new in Windows 2000 and Windows XP. Using quotas, you are able to control how much hard drive space a directory and its subdirectories (folders and subfolders) can use. It allows you to control how other users on the PC utilize hard drive space. This will allow you to limit just how much storage space your son can consume for his mp3 files, or keep a family member from stocking up three years worth of e-mail messages.

Smaller Clusters

Since NTFS can support more clusters than FAT32, the result is NTFS clusters are generally smaller and more efficient. XP and 2K have a default cluster size of 4KB. But if you are converting a FAT file system to NTFS, you will have 512-byte clusters. This may cause you to have a slight performance hit.

NTFS Security

Support for file and directory permissions is one of the major benefits of using NTFS. This security feature insures that only certain users have access to specific files and directories. For example, you can grant Bill complete access to a directory, while preventing Mary from ever seeing its contents.

The security for Windows XP Home Edition is a bit different, NTFS only provides what is known as simplified security. Provided you have set up multiple password-protected user accounts, you can mark certain directories as private. Other users won't have access to your private directories.

The information about file and directory permissions is stored in the Security Descriptor file attribute. When you try to access a file, XP will look at who is logged on; then compare that information to the information in the Security Descriptor file attribute for the file. If it finds you have permission to access the file, it will look up the location of clusters containing the file and open it for you. If you don't have permission to access the file, XP will let you know.

Which File System Is For You?

When you install XP, you will have to decide whether to install NTFS or FAT32.

FAT32 is most useful on multi-boot machines (PC's with more than one OS installed that let you choose which OS you want each time you boot) when file system compatibility is important. If you use a multi-boot computer and want to keeps things simple, install Windows XP on a FAT32 partition. If you are like me and want to play with NTFS but still need to maintain some file system compatibility, you can install XP on an NTFS partition and create a separate FAT32 partition to use specifically for sharing documents between your multiple Operating Systems.

If XP is going to be the only OS on your PC, try using NTFS. In most cases, NTFS will be the most efficient choice and, it offers a range of advanced features which are not present in FAT32.

Converting A File System

If you have already installed your Windows XP on a FAT16 or FAT32 partition and now want to upgrade to NTFS, the convert.exe utility will let you do just that. To convert a partition:

1. Click on the Start button.
2. Highlight All Programs, Accessories, and click on Command Prompt.
3. Type convert.exe c: /fs:ntfs. This will convert your C drive (replace "c" with the proper drive letter to convert another drive).
4. XP will prompt you to enter a volume name. The name you enter will appear next to the drive in My COmputer.
5. If you specified the partition that has XP installed on it, the utility will ask if you want to convert the next time you boot the system. Click Yes and reboot.

The conversion process will create the MFT and other metadata files in free space and won't overwrite any FAT clusters until the NTFS file system is in place. This means if you experience an error during the conversion process, the system should fall back to the FAT file system without losing any data. It also means that you'll need some free space to convert an existing FAT partition. The exact amount of free space required varies according to the size of the partition and the amount of data it contains.

Please be aware that converting to NTFS is a one-way trip. Once the conversion is complete, the only way to return to a FAT file system is to reformat the hard drive, thereby destroying any existing data in the process.

This material is copyrighted by Tom Bair and may not be distributed in part or in whole to any form of media without the express and written consent of the author and/or SFDC staff. Neither the author or Security-Forums Dot Com assume any responsibility for any damages to property or life as a result, or in connection with; the content of this article. It is provided "as-is" and is intended for educational purposes only. All rights reserved, including the right to cover any and all plagerists in peanut butter and lighter fluid and ignite.

Last edited by Tom Bair on Fri Aug 27, 2004 9:34 pm; edited 1 time in total

All those TiB but FAT32 has 2 stings in its tale. The maximum file size is 4Gib but just try & copy it & you find it won't because the maximum copy size is 2Gib aaagh.
Windows, according to the back of HDD packets, has a HDD limit of 137Gb

In theory, the maximum NTFS volume size is 2^64 clusters minus 1 cluster. However, the maximum NTFS volume size as implemented in Windows XP Professional is 2^32 clusters minus 1 cluster. For example, using 64-KB clusters, the maximum NTFS volume size is 256 terabytes minus 64 KB. Using the default cluster size of 4 KB, the maximum NTFS volume size is 16 terabytes minus 4 KB.
Because partition tables on master boot record (MBR) disks only support partition sizes up to 2 terabytes, you must use dynamic volumes to create NTFS volumes over 2 terabytes. Windows XP Professional manages dynamic volumes in a special database instead of in the partition table, so dynamic volumes are not subject to the 2-terabyte physical limit imposed by the partition table. Therefore, dynamic NTFS volumes can be as large as the maximum volume size supported by NTFS. Itanium-based computers that use GUID partition table (GPT) disks also support NTFS volumes larger than 2 terabytes.

The article was a very good read. Does anyone have any filesystem performance benchmarks [Hdtach / Sisoft Sandra] for NTFS and FAT32 for different partition sizes ? If yes then I would be very much interested in knowing the numbers. . One More thing. Has anyone tried adjusting the cluster sizes, etc for improving file system throughput ? If yes again numbers would be welcome.

A question about MFT..do you know of any software that allows one to "see" what is going on in there? From a forensics standpoint, it is a gold mine, but I have searched for the last few months to find something, but no luck. I can check it in any hex editor, but it doesn't present it in any clear manner...

I have some suspicions about the information that may be contained in there, and I would like to poke around more.