How to Break the 2TB (2 TeraByte) File System Limit

How to Break the 2TB (2 TeraByte) File System Limit

In 2006, a RAID array linking together multiple 500GB drives was required to reach the 2TB (2,000 GB) limit and operating systems were just starting to implement solutions to break through this barrier. Now, with single hard drives exceeding capacities of 4 TB (4,000 GB), the need to break this 2TB limit is extremely common, and your operating system may not be able to handle it. If you don’t plan ahead, your operating system will only be able to address the first 2 TB and all that extra storage beyond 2 TB will be unusable. Here is an overview of some of the methods you can use to get around the 2 TB limit.

Operating System Requirements:

Windows XP 32-bit: If you are using Windows XP 32-bit, you are out of luck for native OS support. XP was designed well before this barrier was approached and was not designed to exceed it. Some drive manufacturers offer driver software to allows the full capacity of large drives to be accessed under XP 32-but (Hitachi GPT Disc Manager, Seagate Disc Manager), but user beware. This support is not native to the operating system and can lead to reliability and data recovery issues. Your best bet is to upgrade to a modern operating system.

If using the disc as an additional data drive, either the 32- or 64-bit versions of Vista/7/8 can be used.

The 64-bit versions of Vista/7/8 are required to use large drives as the boot disc (the computer must also be equipped with an newer-style EFI / UEFI BIOS.)

Mac OS X 10.6 and later is required to access large volumes. GUID Partition Tables (GPT) are required.

Linux requires kernel version 2.6.x and later (32-bit CPU limit of logical volume size limit is 16TB, 64-bit CPI limit is 8EB.) (Also, the kernel must be compiled with CONFIG_LBD enabled, which is almost always the default.) GUID Partition Tables (GPT) are required.

Breaking 2TB Option 1 – Use Appropriate Version of Windows with NTFS and GUID Partition Tables (GPT) partitions. It is possible for Windows to use NTFS partitions larger than 2TB as long as they are configured properly. Windows requires that the GUID Partition Tables be used in place of the standard Master Boot Record (MBR) partition tables. You will need Windows XP x64 Edition or Windows Server 2003 Service Pack 1, Windows Vista, Windows 7, Windows 8, or later for GPT support. (It is possible to mount and read existing GPT partitions under Windows XP and 2000 using GPT Mounter from Mediafour.; however, their MacDrive product does not support GPT partitions.) There are a few stipulations for GPT disks:

First, the system drive on which Windows is installed can’t be a GPT disk because it is not possible to boot to a GPT partition unless you have the 64-bit version of Windows Vista/7/8 and a UEFI system BIOS.

Breaking 2TB Option 2 – Use Linux with CONFIG_LBD enabled. Most Linux file systems are capable of partitions larger than 2 TB, as long as the Linux kernel itself is. (See this comparison of Linux file systems.) Most Linux distributions now have kernels compiled with CONFIG_LBD enabled (Ubuntu 6.10 does, for example.) As long as the kernel is configured/compiled properly, it is straight-forward to create a single 4TB EXT3 (or similar) partition. In general, this is applies to Linux kernels 2.6.x and later.

Breaking 2TB Option 3 – Use Standard Partitions and Create Multiple Volume Sets within a RAID array. A RAID array itself can be larger than 2 TB without presenting a volume set larger than 2 TB to the operating system. This way, you can use older file systems (that support only 2TB) and still have RAID 5 protection and more than 2 TB of total storage. To do this, put all 5 drives into a RAID set and create a 2 TB RAID Level 5 volume set — this will leave 2TB of the RAID set unused. Then create a second 2 TB RAID level 5 volume set. Boot into your operating system, create a partition on each of the 2TB virtual drives, and format each of the two 2TB virtual drives. The disadvantage is that there is not one single, large 4TB partition. The advantage is that 1) backwards compatibility for the file system and partitions and 2) they are both part of a RAID 5 array and are protected from single drive failures and only 1 drives worth of storage is sacrificed for RAID parity data.

Several hard drives to connect to the RAID controller to create a RAID array. For this example, I’ll assume (five) 1000GB SATA drives.

Drives must be configured in a RAID level 5 Volume Set. For the first two examples, I’ll assume all five drives are members the same RAID level 5 volume set. RAID level 5 requires the space of 1 drive to be allocated for parity data, so total available storage space for 5 drives will be 4 drives x 1000GB = 4TB.

RAID Required Background Information: I’m assuming you already have an understanding of RAID 5, its benefits, and requirements. If not, read this Wikipedia article. Now, let’s discuss the difference between RAID sets, Volume Sets, and Operating System Partitions.

RAID Sets are groups of drives that a RAID controller groups together to act as one single array. The individual disks are not visible to the operating system but rather are controlled by a hardware RAID controller.

Volume Sets are create by the RAID controller and reside on top of RAID Sets. A Volume Set set is presented to the operating system as a single, virtual disk drive. This is a little confusing, but the RAID level (RAID level 5 in this example) is determined when the Volume Set is created (not when the RAID Set is created.) It is possible to have multiple Volume Sets residing on the same RAID set, and the Volume Sets may even use different RAID levels.

Partitions are created by the Operating System and reside on top of Volume Sets. (Volume Sets appear as virtual disk drives to the operating system.) You can use have the Operating System create one or more formatted partitions on top of a volume set.

Note 1: RAID Capacity Expansion. If your RAID card supports online capacity expansion, it is possible to expand any of the configurations above. For options 1 and 2, expand the RAID Set, then Expand the Volume Set, then Expand the Operating System partition. For option 3, expand the Raid Set, Create a 3rd Raid level 5 Volume set, and then create a third operating system partition. To learn more about expanding a RAID array on an Areca controller running Windows, read this article.

Note 2: Software RAID. Software RAID adds an additional level of complexity to RAID. For that reason, I recommend using a Hardware RAID controller. Having said that, I think everything mentioned above is technically possible if you are using software RAID, but I’ve never messed with it some I’m not positive.

106 Comments

I would be very interested in anyones views on my problem.
Like quite a few people i use multi-boot (windows vista 64bit ,vista 64bit and XP 32bit) on 3 WD Velociraptors. I also have 6 x 2t in the pc and 24 x 1tb in 2 external esata towers.
What i need to do is to create 3 internal 4tb raid volumes (striped) and 4 external 4tb raid volumes (striped) NO RAID 5.
this will be presented to windows as 7 x 4tb drives.
Now this is easy under Vista & Windows 7 but as i multi boot i need these to be available to windows xp 32bit also.
My external towers are raid, so is my mother board and i have LSI SAS 3041E-R 4-PORT SATA RAID PCI EXPRESS CONTROLLER available to me.

So i think there is no problem creating the raids but how do i make 4TBs that work with ALL my multiboot systems??

Steve, I think what you are wanting to do is not possible with older operating systems such as XP 32-bit (it’s 2001-era technology.) The only way to get close to this is to create an XP virtual machine and have it go through the host operating system to get to the volumes. Microsoft Virtual PC and Sun VirtualBox are 2 free solutions. Otherwise, I don’t believe it’s possible.

Hi Carlton. I do have VMWare and Vitualbox on the 64bit boot drives so I will try those out.
FYI : I have found one way to achieve what i wanted, using only built in windows utilities but the speed is not acceptable. Mainly involves using dynamic disks and spanning them.
Thanks

I don’t think that is the problem that Carlton is refering to. Specifically, you can’t access over 2 TB of data using a 32 bit operating system. The problem is that 32bits is 2 TB.
In order to address more than 32 bits (2TB)in windows XP, you have to insert a method to page hard drive space in and out.
That is effectivly what a disk partition does. It allows you to go above the limit by adding another addressing “bit” in the form of a drive letter. But hoping for XP to natively support more than 32 bits on a 32 bit operating system is not going to happen….

Calvin,
I have a CCTV video recorder that is running Windows XP 32bit. I am trying to increase my record time by adding a 4Tb hard drive. I don’t think speed will be an issue because it is just video data storage space. I am trying to partition the drive as two separate 2Tb drives but when I do this the hard drive is no longer recognized by Windows. Most of what I read about the problem of using hard drives larger than 2Tb on XP 32bit is that people don’t want to make their large drive into two smaller ones. I don’t think this is a problem for me, the DVR will just use the space like any other drive. Is there something special that I need to do to get windows to recognize the drive?

Mark, XP 32-bit can’t address the 4TB drive at the base hardware level. Putting partitions on top of hardware that can’t be fully addressed is never going to work. XP is 13 years old. . . so best bet is to use older hardware or update to a newer OS.

The trick is to uses Windows XP 32bit and NTFS BUT not use Windows 32bit to partition and then format the drive.

1) When we say 32bit or 64bit versions of windows we are referring to how much addressable memory the o/s can see/use/read/write which has nothing to do with disk drives. Indeed 32bits is 4GB of addressability NOT 2TB.
2 )Surprise !! Windows XP 32bit can use hard drives (and prtiotions) well beyond 2TB (thats a direct copy from Microsoft). The 2TB limit isnt in windows, its the use of MBR partitions by 32bit Windows which causes the restriction. XP 32bit can also use FAT32 etc and each of those has its own limit too but no one seems to be claiming those as limits anymore.
3) 32bit windows does not support GUID partions and so the myth of windows itself having a 2tb limit has been born.
4) if correctly configured windows 32bit can actually read (only) 4TB GPUID drives but not create or write to them.
5) additionally NTFS has no 2TB limit, and windows is quite happy to use non-microsoft partitioning tools to create > 2TB volumes/partitions that NTFS is happy to format under 32/64bit windows.
6) one more common way is to use a raid card with utilities to make larger partions (not MBR or GPUID) and format them with NTFS. These partions can not be created with windows 32bit, but work just fine for read/write under 32bit windows becauses its how the drive is presented to windows that counts, not how many bytes are involved.

I respond to questions; this is apparent from the comments on this post. If you have a question or suggestion, please post it. If you want to complain about the information I provide for no other reason than to help others, please don’t.

This post explains that you need GPT to access volumes over 2 TB in Windows. If you want to know “how to format GPT” then, click the link at the end of the GPT paragraph for “Read this Microsoft TechNet article for more details on GPT.” It explains GPT, mentions diskpart, and links to pages explaining how to use it as well as the Disk Management Console. I provided the link with the information you needed at the end of paragraph you were referencing.

This post is not intended to be an instructional on how to use Windows, how to configure specific RAID controllers, or how to use software RAID within Linux. All of those topics are mentioned and each would warrant a unique post to even start to explore them. Microsoft Technet does a great job of documenting Windows; I see no need to reproduce that when I can simply link to it instead.

I saw this reply in your thread and chuckled… It’s funny, I was down the same path as Davinci and I figured it out on my own, with some help from Carlton, but I didn’t b1tch about researching. It’s called knowledge for a reason, some one can’t give you that, you have to learn IT. geeesh. I give you mad props Carlton for dealing with crap like this.

Oh, btw… with your kind words, and a gentle nudge in the right direction, I was able to find the research I needed to help me build out 24Tb. Quite nice, but I am now looking into a better backup system. =p

Ok Maybe I should not have bitched so much but it gets frustrating reading article after article on the subject that goes into a long diatribe about info you do not need. You get to the point where you want to strangle someone for failing to provide the HOW! You begin to contemplate are you the only one who wants to know how to accomplish a task instead of understand the technical details of the outcome?

How hard is it to say to enable GPT you need to create 2 drives on your raid controller one to store the OS and the other for data as the OS can not be easily installed on a GPT drive. Once you installed Windows run diskpart or use the windows computer management, select “disk management” then right click the section that says “drive 1” and select “Convert to GPT Disk”.

Basically, I want to create a partition with something like 80GB for Windows 7 x64 with RAID 10 spanning 4 1TB disks. Leave the rest of unparititioned volume alone for now. Then, in the operating system, I want to partition the other volume to GPT. In doing so, I should have 2.6TB of RAID 5.

That is correct. After created 2 logical disk drives in your RAID controller one with 80GB the other with the rest of the space you can open up disk manager you will see your 80GB DRIVE (not a partition) for the OS. Below that will be another DRIVE split into 2 partitions, one is 2TB the other is the rest of the space. Right Click the *text* “Disk 1” and select “Convert to GPT” and the split will disappear.

Great article. Thanks.
No problems finding the diskpart information, but was even easier to accomplish in the Windows Server 2003 disk management tool.

I used the general Windows GPT with a slight twist, since I wanted two partitions, one for system/OS other for data, on a single large RAID6 set using a Promise Fastrak hardware RAID card. So, I made one RAID6 volume of 100GB and a second RAID6 volume (both striped across the same set of 6 x 2TB disks) of the remaining 7.9TB. This way the OS sees two seperate logical disks – one partitioned as MBR for the OS boot, the other converted to GPT for a larger data disk.

Originally, I setup one large RAID volume, but as soon as I created the first OS-level partition for the OS installation on the logical drive, I could not find any way to make the remaining space a GPT partition.

BTW…
Davinci: The other 6,692,030,276 entities inhabiting this planet are not here specifically to provide for your needs. It might be nice to thank someone who takes the time and effort to write out even “part” of what you wanted to know.

Jay said:
“Davinci: The other 6,692,030,276 entities inhabiting this planet are not here specifically to provide for your needs. It might be nice to thank someone who takes the time and effort to write out even “part” of what you wanted to know.”

to Mr. Unrein question – the built in array controller in the msa doesn’t support logical volumes over 2TB. Sadly. Use the Compaq/HP ACU array config utility to configure your logical volumes (with raid) (looks like you’ll get 3 logical disks 2, 2tb and maybe 1tb?) then use Microsoft’s disk management mmc to create your spanned volume. Voila, 1 drive letter and about 5tb of usable fault tolerant space. The fault tolerance is what you chose during the logical drive setup. So if you choose raid 5, then you can lose 1 drive. If you do raid 6 you can lose 2 drives.

I told you that so I can ask this: using the scenario as Mr. Unrein’s here, I want to extend the space on my MSA which uses sata drives connected to a Compaq Array controller 642.. On a different array controller, a 5300 series controller from Compaq are SCSI disks. I created 1 logical disk, raid 5. In the disk management console I converted the disk to dynamic. What issues does anyone see with extending the volume on to this dissimilar hardware?

Hardware and Software raids have their own pros and cons. While hardware raids are faster, a failure of the controller card means searching for a new one, and controller chips change very often. It may be very hard to find a replacement controller that can just accept your existing raid drives. Software raid avoids that issue, but there is a performance hit since your system processor is needed for parity calculation and that does take cycles regardless how optimized. I have 4x 2TB drives in a Radi5 array on Ubuntu Server, using the mdadm software raid, and it’s working just fine. The performance hit doesn’t matter since all that space is used for network shares.

In short, you need to know the pros and cons and decide for yourself what risks are worth what benefits.

It wasn’t designed that way for legacy purposes. It was the best solution at the time given the available technology at the time. It became an issue when technology surpassed the limitations of the specification.

DISKS BIGGER THAN 2TB WITH UNIQUE PARTITION, FORMATTED AND WORKING SOLVED

Hi. An easy way to get partitions over 2TB formatted and working.
When you are about to create or format the new partition in windows 8/7
Delete all partitions in the disk bigger than 2 TB
Hit shift+F10 and the command line will open.
Then write “diskpart.exe” ENTER
Then write “list disk” ENTER
Then “select disk X” ENTER – X is the number of the disk bigger than 2TB
Then “convert GPT” ENTER
Until this point is what microsoft supports tells you, then you are on your own
if you don’t proceed further creating the partition from diskpart, probably the windows setup will still not be able to create partitions bigger than 2TB

To make sure windows is able to see a partition bigger than 2TB you need to proceed like this:

“create partition primary” ENTER
exit
exit

Now the new over 2TB partition is created and you can proceed formatting from the windows setup.

Many thanks guys, I was trying to set up a 64 bit windows 8 disc to setup a single 4tb drive, to run the os wasn’t working at all, would create 2 2.1gb partitions, then I thought id use windows 7 disc (64 bit) instead of windows 8- worked fine! not sure if its the disc I got ( which is labelled 64bit) or actually a wider problem with windows 8- but it worked fine for windows 7- give this a try if windows 8 isn’t working!

Hi carltonbale or other… I have an old system with windows XP and an 1.5 TB Hard Drive that boot on it, But I begin to run out of space. I would to buy another hard drive greater that 2 TB like 3 TB or 4 TB. Is it possible to use as Storage disc ? By the way I have not isues to use more that 1 portitions for the disk. I gonna change my config at the end of the year but for now more space will be appreciate.

Windows requires that the GUID Partition Tables be used in place of the standard Master Boot Record (MBR) partition tables. You will need Windows XP x64 Edition or Windows Server 2003 Service Pack 1, Windows Vista, Windows 7, or later for GPT support. (It is possible to mount and read existing GPT partitions under Windows XP and 2000 using GPT Mounter from Mediafour

Now a new over 2TB unique partition is created and you can proceed formatting from the windows setup screen.
Then you can keep installing windows in that partition or just exit setup to keep your new partition
clean, empty and ready to get filled up.

This is what I was thinking myself, as I am about to buy a 2TB HDD as part of a final upgrade (from 80GB!) for an aging but happy Windows XP x86 (32bit) system. Most of the HDDs on offer seem to be 3TB for best bang-per-buck.

The inability of geeks to translate something complex into something simple, is something that annoys me. However, I also suffer from it myself! Also annoying is dumbing-down and not giving ENOUGH information to make an informed and safe decision on a technical implementation, which at least this article generally does…

Still, I think this article’s headline makes people find it in a web search who are looking for a simple Windows XP 32-bit workaround for the 2TB MBR-partitioning limitation.

Which apparently doesn’t exist. It might have been a good idea if this had been realised and mentioned in the article, regardless of the main target audience.

When this article was written back in 2007, the focus was on using RAID to even reach the 2TB limit. With larger drives now readily available, RAID is obviously no longer required. So I updated the article to reflect this and give more examples and OS restrictions. You can use XP 32-bit with driver software from the drive manufacturer, but it’s much better to upgrade to a modern OS with native support.

Ive scanned threw the replies and not sure i found the answer i was looking for. in a nut shell can a larger than 2tb drive be formatted in a machine with a 64bit OS than put into a machine using a 32bit OS? (as a storage drive) the OS in my case is *cough* Vista

Firstly, thanks for the helpful article… I’m looking into all of this and just when I think I have understood, I get confused again.

I am running Windows XP SP3 and want to increase hard drive space. I do NOT need a large HD for the OS/Boot (I have a 500GB and it does fine). My new HDs would be for data (mainly FLAC files for music playback) and back up.

1. As I understand it, I can get some manufacturer’s 3-4TB external drives and they will work with XP fine. That’s because there is an interface in the drives which converts the sector size from 512bytes to 4096. But if that interface breaks, the data on the drive will no longer be accessible. Is that right?

2. Perhaps I have got this wrong, but is the change in sector size from 512 – 4096 (or at least “apparent size” as recognised by the OS) known as “Advanced Format”?

3. If so, if I buy a 2TB “Advanced Format” hard drive, what issues if any will there be for me? If possible please answer separately for internal and external HDs if the answer is different for them.

4. What’s all of this about using realignment software. I’m confused as to when I would need to do that and what the implications are.

Sorry for the many questions and thanks a million to anyone who is able to help.

Max, converting sector size and reformatting is different than XP being able to address >2GB. Advanced Format is the larger sector size, not the conversion from one size to another.

If you buy a 2TB advanced format drive, you will need to switch it to the smaller sector size. Instead of doing an alignment, there is probably a jumper on the drive to select 512byte sector size. Install the jumper and then reformat the drive. That’s it, no special software. There will be a slight performance decrease, but not enough that you’d notice. This is what I recommend.

Internal and external drives are handled the same by the operating system. I’d recommend internal because it’s easier to get to the jumper to switch sector size.

Realignment software is for Advanced Format drives that don’t have a jumper to put them into 512byte sector size mode. I’d avoid messing with this. Get one with 512 compatibility mode.

Lastly – it’s time to upgrade your OS! XP is end of life and security issues are no longer being addressed. Windows 7 is an easy switch. Windows 9 will be here in a few months.

I already understand and have accepted the 2-TB limit, with XP. I’m getting ready to build another Video H/D Editing Unit, using Win7-Ultimate 64-bit. A Gigabyte: GA-Z97X-Gaming7 m/b, is my choice. It has 8-“Sata3” connectesions on board.
My 1st question: “Is it the O/S specifically, that limits and/or recognizes the size of the internal drives”? My 2nd question: “Does the design and/or specifiations of the M/B, have any bearing on the size of the drives connected to it”?
I would like to purchase and use “FULLY”… six (6) 4-T/B – 7200 WD-Black, as my storage drives. I know that there is no problem, with Win7-64 O/S recognizing them as external drives. I’m doing that now, with an earlier built Win7-64 unit.
THANK YOU, Jerry S. New York

Jerry, Going above 2TB requires both the operating system and the hardware be capable. Pretty much all motherboards made after about 2008 are capable, so you shouldn’t have any issues as long as you use a modern operating system.

On an old computer from 2007, is it possible to split a 6TB drive into 2TB partitions and fill them with data using windows XP Pro x64 only? Or is it something that my controller or OS wouldn’t understand?

In short, does the OS / controller consider the whole 6TB drive when writing data or will work fine with 2tb or lower partitions no matter how big the drive?

32-bit XP can support up to 16TB drives. The cluster size simply needs to be changed from 512 to 4096. Nothing else needs to be done. The only limitation is that it cannot be used as a boot drive.

It’s not an MBR or NTFS or Windows restriction. It’s just a cluster setting limitation. Even FAT32 can support 16TB drives(you just have to split it into 8 pieces).

Now there is technically a limit to what size a partition can be in 32-bit windows. No partition can be over 2tb. So if you have a 10TB drive, you will need to split it up into 5 partitions. You can’t do this with a native Windows application, but there are a lot of programs that can do this. Almost any partitioning program like Partition Magic, EaseUS, Paragon, and Partition Guru can all do this.

In theory, 32-bit XP should be able to access up to 256TB drives(by setting cluster size to 64kb), though the highest I’ve ever formatted is 10TB. None of this requires RAID or Dynamic volumes.

@Wampus Hunter Thanks for the cluster size info. It’s another way to address larger partitions. The original intent was how to do this within Windows and RAID controllers. When this post was originally written, 500GB was the largest drive size available and there were no third-party partition tools. What you’re describing is basically what I described with the RAID volumes and the RAID controller doing the partitioning. The larger cluster sizes weren’t RAID-friendly at the time. Now that 10TB drives and third-party partitioning tools are available, that’s definitely a good option.

But really, the best option now, is to switch to a modern operating system. Windows XP was a fantastic operating system for it’s time, but that time has unfortunately passed.

Hey Carlton. I have been working on this problem for some time and even posted my problem everywhere but no one has a specific answer to me. Please help. I have a toshiba qosmio x505 w898 gaming laptop that runs windows 7 Ultimate (I just upgraded since someone said it may help with my problem) 64 bit. I have two hard drive ports. The first one has a samsung 2 tb ssd. The second port is a samsung 4 tb ssd that I use for data only. But windows will not let me use all 4 tb. I had it initialized and formatted to 4 tb. Windows, bios and samsung magician software all see it as 4 tb but when I try to use all 4 tb, the drive becomes unreadable and corrupted. I tried installing new copies of the files (itune movies and tv shows) by downloading it from itunes.

Some background. My windows will not let me initialize or format it to 4 tb. I had to install the 4 tb ssd in another computer and have it do it. Then install the ssd back into my computer. Now my windows sees it as 4 tb. But when adding files, it gets corrupted.

I went to bios and there is not an option to switch it to uefi. So i’m stuck on bios. I installed alllllll the updates. Bios and also the intel driver per the suggestion of some of these computer posts. Still I cannot use my full 4 tb.

I did partition the 4 tb into 4 parts. about 1 tb each or 930 gb each. But it gets corrupted or unreadable when I add more files to use the 4 tb. Help!!!!! Please.

I really wish I could help you out, but you’ve already tried everything I would suggest. There’s some type of hardware defect that software is not going to be able to fix without some really messy workaround that I can’t think of.

You seem to have a solid understanding of drives and the way they work, so I wanted to present an idea here to get your thoughts on.

Today you can buy 8TB hard drives. But what if you wanted a drive like this to be usable with an older system such as xp 32-bit. There’s two problems here, and I know how to deal with one of them–the 2tb partition limit for fat32–but the recognition of the drive itself is what I’m a bit fuzzy on.

I know proper drive recognition will be dependent on the bios as well as operating system. Assuming that the bios is good, would a 32-bit operating system have an issue with an 8TB drive?

And some related thoughts–how would such a formatted drive (8TB into 4 2TB partitions/volumes) work as an external drive? How about as a network drive?