About The Author

19 Comments

2003-11-10 7:45 pm

So what makes anyone think that the TIA (Total Information Awareness) project was ever really dropped, even if officially?

2003-11-10 8:04 pm

Yes, please note that this only discusses PUBLICLY KNOWN databases. I seem to recall seeing a 16 TB tape library unit from the _1980’s_ in the NSA’s Cryptography Museum. I would be very surprised if the worlds largest database wasn’t located in nearby Ft. Meade, and isn’t at least an order of magnitude larger than what this article discusses.

-braddock

2003-11-10 8:11 pm

From the article…

The program requires that the databases must be in production and contain at least 1 terabyte of data (or 500 megabytes of data if running on Windows).

Why special considerations for windows???

2003-11-10 8:37 pm

Aren’t you supposed to be “Missing In Action”..?! Sorry, couldn’t help it!!!

2003-11-10 8:42 pm

“”Why special considerations for windows???””

Processor limitations?

Windows implies x86 hardware, which comes with a 4Gb (Without using paging extensions) address space, afaik Windows uses the top 2Gb for kernel addresses, with the rest user space sections. So maybe Windows applications are limited to 500mb of data area (Perhaps someone more knowledgable can clarify this).

Even Linux is limited to 3Gb of application address space on x86 (Top 1Gb is used for the kernel, same PAE caveat as before). So trying to run a terabyte sized DB on x86 architecture involves a lot of shuffling around of data, which would slow everything to a crawl. So perhaps they’re trying to be fair in comparison to processors with a 64-bit address space.

2003-11-10 8:55 pm

I have seen numbers indicating that on average 20% of all data is incorrect due to poor data-entry, fake data etc.

Just think about it – 20% of 30 Terabytes is 6 Terabytes of useless/garabage data. Wow !!!

2003-11-10 8:55 pm

Largest database:

Babar:

This database is arguably the largest in the world: as of Mon Nov 10 00:01:28 2003, over 834.1 TB has been stored in 710021 files.

Oh well i’m having 4gb databases atm with mysql with no problems at all, I will tell you when I reach 30TB ok?

2003-11-11 9:22 am

MySQL has troubles with anything over 30MB, let along 30TB.

Some peoples run multi-TB database on MySQL. My database is “only” 230 MB and is running without a hitch. If you have a problem with 30 MB, then you misconfigured something – don’t blame MySQL for it.

2003-11-11 10:24 am

Let’s not forget that MS is also distributing a version of Windows specially for powerful servers (Datacenter Edition). My guess is that 500MB is a typo and that they really meant 500GB.

2003-11-11 1:33 pm

The myth that MS SQL server can’t compete in the large database arena gets blown out of the water! a 5.3 TB SQL server 2000 database..thats pretty impressive. I would still like to see the hardware setup for it.

2003-11-11 5:39 pm

Hardware setup – simple.

A large Sun box with an “Intel Inside” and a “Designed for Windows” badges tacked on with scotch tape.