Monday, June 10, 2013

A Note on PRISM and IOPS

I have several posts half-written on the subject of the NSA's data-gathering efforts, but let me start with a few words on which things in the computing world are and are not subject to "Moore's Law".

Let's first say that the real Moore's Law is a narrow statement about the rate of increase in the number of transistors per integrated circuit, but in its colluqial usage means something like "computing power per unit cost increases exponentially, doubling about once every 18-24 months" That is, if I buy $1 million worth of computing power today, and then 2 years later spend another (inflation-adjusted) $1 million on computing power, the newer computer will be roughly twice as "powerful" as the older one, for some definition of "powerful".

Moore's law cannot make this platter spin any faster.

Now let's talk about the fact that certain things happening inside your computer are physical processes that researchers have had a hard time getting onto the Moore's Law growth curve. Most importantly, large hard drives consist of platters that must be spun at higher and higher speeds in order to increase disk read & write performance.

Network bandwidth appears to be on a Moore's law-style curve, but the rate of growth is somewhat slower than, say, the growth in the amount of RAM you get for $100 or the amount of CPU power you get for $1000.

These facts turn out to be important because the amount of data supposedly being collected is large enough that it has to be kept on hard drives, as opposed to in RAM or even on solid-state drives. This has important consquences for our ability to do meaningful research on the raw data. But we'll cover that later.

who we are

Nicholas Beaudrot is an accidental political observer living in Seattle, Washington. By day he writes software for Amazon.com, snowboards, and plays ultimate frisbee. By night [and morn] he posts to this blog, runs the Seattle chapter of Drinking Liberally, and tries to cook decent Italian cuisine. A graduate of Brown University with a joint degree in Mathematics-Computer Science, in late 2003 Nicholas felt the urge to put his knack with numbers towards a greater social purpose than winning his fantasy baseball league or taking up poker, perhaps in an act of penance for not voting in 2000. He has been spotted standing in line for Star Wars Episode II: Attack of the Clones, on the Atlanta area quiz bowl program "Hi-Q", and as a young boy in national broadcasts of the Christmas Eve service at the Cathedral of Saint Philip. If you play Halo 3, Team Fortress II, Rock Band 2, Catan, or a number of other games, he's on Xbox live as niq24601.

Neil Sinhababu is a philosophy professor at the National University of Singapore. It's a tropical island with good public transit and they're very nice about not caning him. He's fond of red-state college towns like Austin, where he got his PhD. Much of his research is in ethics — hence his alias "Neil the Ethical Werewolf," which contains the name of his philosophy blog. He has also published on Nietzsche and on how to have a girlfriend in another universe. His utilitarianism shapes his goals and tactical views, and makes it impossible for him to stay away from politics. At Harvard, he won a student government election by eating fire in each dorm room in his district. He'd be happy to use this skill to help Democrats in tough races. He likes drinking with smart people and dancing in altogether ridiculous ways. At his last project, War or Car, he showed that you could buy each US household a Prius or each panda a stealth bomber for the price of the Iraq War.