Monday, May 24

The Life And Death And Life Of Moore's Law

I've written recently on the untimely death of Moore's Law and on one of the first side-effects of the faltering and failure of that law. But, being somewhat dead myself, I didn't have the time or energy to go into any detail, and probably left my less-geeky readers saying something along the lines of Huh?

But this is important, so I'm going to give it another try.

Way back in 1965, just four years after the first integrated circuit was built, Gordon Moore, then working at Fairchild, made an observation and a prediction.

His observation was that the number of components in an integrated circuit was increasing, while the cost of each component was decreasing; his prediction was that this trend would continue. Intel has made his original paper available for you to read. It's a little bit complicated; Moore is talking about trends in the number of elements in a integrated circuit required to achieve the minimum cost per component - efficiencies of scale, in other words.

Reduced cost is one of the big attractions of integrated
electronics, and the cost advantage continues to increase as
the technology evolves toward the production of larger and
larger circuit functions on a single semiconductor substrate.
For simple circuits, the cost per component is nearly inversely
proportional to the number of components, the result of the
equivalent piece of semiconductor in the equivalent package
containing more components. But as components are added,
decreased yields more than compensate for the increased
complexity, tending to raise the cost per component. Thus
there is a minimum cost at any given time in the evolution of
the technology. At present, it is reached when 50 components
are used per circuit. But the minimum is rising rapidly
while the entire cost curve is falling (see graph below). If we
look ahead five years, a plot of costs suggests that the minimum
cost per component might be expected in circuits with
about 1,000 components per circuit (providing such circuit
functions can be produced in moderate quantities.) In 1970,
the manufacturing cost per component can be expected to be
only a tenth of the present cost.

The complexity for minimum component costs has increased
at a rate of roughly a factor of two per year (see
graph on next page). Certainly over the short term this rate
can be expected to continue, if not to increase. Over the
longer term, the rate of increase is a bit more uncertain, although
there is no reason to believe it will not remain nearly
constant for at least 10 years. That means by 1975, the number
of components per integrated circuit for minimum cost
will be 65,000.

I believe that such a large circuit can be built on a single
wafer.

What he's saying is that by 1975, it would be cheaper to build a single integrated circuit with 65,000 components than to build two 32,500-component circuits - and, by comparison, a 130,000-component circuit (if such a thing could be built) would cost more than twice as much.

Events since then have proved him right (and happily he is still around to enjoy it). And more right than he imagined because not only have the components been getting smaller and cheaper, but at the same time they have been getting faster and using less power. And this has been going on, following a curve where (to take the most famouse example) processing power has been doubling every 18 months. For my entire life processing power has been doubling roughly every 18 months.

My first computer, which I bought as a teenager, saving pocket money every week until the day of the Big! Christmas! Sale! was a Tandy (Radio Shack to many) Colour Computer. It had 16k of ROM (which contained the BASIC interpreter; there was no operating system as such) and 16k of RAM. It was powered by a Motorola 6809 processor and a 6847 video chip. It had a maximum resolution of 256 by 192 - in black and white - or 16 lines of 32 columns in text mode.

It ran at 895kHz.

Yes, boys and girls, kiloherz. It was an 8 bit chip (with a few 16-bit tricks up its sleeve, admittedly); it could execute, at most, one instruction each cycle, and it ran at less than a megahertz. (Also, it had no disk drives at all; everything was stored on cassette tape, which fact is directly responsible for the irretrievable loss of my version of Star Trek and the completely original game Cheese Mites.)

Not quite twenty years on, I'm typing this on a system with a 2.6 gigahertz 32-bit processor than can execute as many as three instructions per cycle, some of which can perform multiple operations like doing 4 16-bit multiply-accumulates all at once. It has more level-one cache than my Colour Computer had total memory. Its front-side bus is eight times as wide and nearly a thousand times as fast. My display is running at 1792 by 1344 in glorious 24-bit colour. And it has six hundred and fifty gigabytes of disk.*

It cost a bit more, it's true. My 1984 Colour Computer cost me $149.95, and Kei, my 2003 Windows XP box, cost me around $2000. The best I can do today for $149.95 (ignoring for the moment two decades of inflation and the fact that this now represents a morning's earnings rather than a year's) is a Nintendo Gamecube. The Gamecube only runs at 485MHz (achieving a measly 1125 MIPS); it only has 40MB of memory; it only has 1.5GB of storage. Its peak floating-point performance is a mere 10.5 GFLOPS, compared to the Colour Computer's... I don't know, exactly, since the CoCo had no floating-point hardware at all, and I doubt that the software emulation achieved so much as 10.5 kiloFLOPS.

So, depending on exactly what you wish to measure, 20 years of innovation has given us somewhere between a thousand and a million times better value for money.

And here it is again: This has been going on for my entire life. Every year, tick tick tick, new and better and faster and cheaper. You buy the latest and greatest and it's obsolete before you get home from the mall. It's so much a part of our lives that it's a joke, a cliche.

Anditjustdied. [That last link goes to an IBM presentation, the first 13 pages of which are just general marketing material, but pages 14 to 24 go right to the heart of the problem.]

The death of Moore's Law has been predicted many times, not least by Moore himself, but when you get IBM's Chief Technology Officer saying

Scaling is already dead but nobody noticed it had stopped breathing and its lips had turned blue.

you know something's up. Particularly when he's not making a prediction, but talking about what's happening right now.

And everything was planned so neatly too. 90 nanometres was to come on line late '03, ramping up this year; 65 nanometres was to be the big thing of '05, followed by 45 nanometres in '07. Now, beyond that, at 30 nanometres and 20 nanometres, things were less clear, and beyond 20 nanometres not clear at all, but at least the path was marked out from the old 130 nanometre stuff down to 45, giving us 9 times the transistors and 3 times the speed. Only someone forgot to check with the laws of physics.

Wired: How long will Moore's Law hold?

Moore:

It'll go for at least a few more generations of technology. Then, in about a decade, we're going to see a distinct slowing in the rate at which the doubling occurs. I haven't tried to estimate what the rate will be, but it might be half as fast - three years instead of eighteen months.

What will cause the slowdown?

We're running into a barrier that we've run up against several times before: the limits of optical lithography. We use light to print the patterns of circuits, and we're reaching a point where the wavelengths are getting into a range where you can't build lenses anymore. You have to switch to something like X rays.

So, what exactly is the problem? It's not, as Moore and others predicted, a question of actually building the circuits - that's still working fine. IBM, Intel, AMD and others have all produced working chips at 90 nanometres. The problem is leakage. Each of the millions of transistors in a chip is a tiny switch, turning on and off and incredible speeds. Each time you turn the transistor on, or off, you need to use a little bit of electricity to do so. That's okay, and it's expected, because you don't get anything for free. The problem is that the transistors are now so small, and the layers of insulation - the dielectric - so thin, that they leak. There's a partial short-circuit, and so instead of only using power when the switch switches, it's using power all the time.

So what? Electricity is cheap. Well, the so what is heat. Modern microprocessors use as much electricity as a light bulb, and that means they produce just as much heat. If they didn't have huge heat sinks and fans bolted onto them, they'd very quickly overheat and fail - a fact that some people have independently discovered.

Until now, each new generation of scaling, each new node, has brought smaller, faster, cheaper and cooler transistors. At 90 nanometres, transistors are smaller, cheaper, probably faster again - but they run hotter. And the competition in the processor market has already driven power consumption (and heat generation) about as high as it can go. So when the new generation was discovered to increase the heat rather than decrease it, the whole forty-year process of accelerating change ran head-first into a wall.

Back at the end of 2002, I made the following set of predictions for the coming year. I felt pretty comfortable in all of them, the first no less than any of the others:

My predictions for 2003:

1. Microprocessors will hit 4GHz by the end of the year. Marketers will try and largely fail to convince the public to buy them.
2. A major scientific breakthrough will lead to a new and deeper understanding of something.
3. A major political scandal will result in a huge media kerfuffle and only die down when someone resigns.
4. There will be a war.
5. Bad weather will affect the lives of millions of people.
6. There will not be any major, civilisation-destroying meteor impacts.
7. Astronomers will find new and interesting things in the sky.
8. Spam, pop-ups and viruses will continue to plague us. The Internet will fail to collapse under the strain. Pundits will predict that this will now happen in 2004.
9. A rocket will explode either on the launch pad or early in its flight, destroying its expensive payload - which will turn out to be uninsured.
10. Cod populations in European waters will continue to fall, and the European parliament will fail to act to prevent this.
11. A new species of mammal will be discovered.
12. A species of reptile or amphibian will be reported as extinct.

But not only did we not see 4GHz processors in 2003, it's doubtful that we'll see them in 2004 either. (I was wrong about number 3, too. No-one resigned, and the media moved onto the next scandal. Rinse, repeat.)

Now, assuming you're not a hard-core computer gamer, hanging out for the release of Doom 3 and Half-Life 2, why should you care?

Well if you have broadband internet, or a mobile phone, or a DVD player, or a PDA, or a notebook computer, or a digital camera (or a digital video camera), or you use GPS on your camping trips, or you enjoy the low cost of long-distance phone calls these days, if you download anime or the latest episode of Angel off the net, if take your iPod with you everywhere you go, if your job or your hobby involves using e-mail or looking things up on the Web, you can thank Moore's Law for it.

Modern communications depend critically on advanced signal processing techniques, performed by specialised chips called Digital Signal Processors, or DSPs. These things are everywhere - every modem, every mobile or cordless phone, every digital camera, every TV or VCR or DVD player, every stereo, every disk drive. It's the relentless advance of Moore's Law that has made DSPs fast enough and cheap enough to do all this, and made them efficient enough to run on batteries so well that your mobile phone might last a week between charging. (My first mobile was lucky to make it through the day.) Disk drives demand high-speed DSPs to sort out the signals coming from the magnetic patterns on the disk and turn them back into the original data. DVD players need them to turn the tiny pits pressed into the aluminium surface into a picture. The entire global telephone network, mobile and fixed, depends on DSPs. And any advances in any of these areas will require more and faster and cheaper DSPs and - uh-oh.

And there's more: The advances in computers and communications over the past four decades have been the primary driver of the global economy. The economy has been growing all that time, even though we have made no fundamental breakthroughs in finding new resources or new materials. If you're better off than your parents, you can thank Moore's Law for a big chunk of that - if not the effort you put in, then the new opportunities it opened up.

And it just died.

I don't think the financial markets have a clue yet what's going on, but in any case it's going to be a soft landing. All of the processor manufacturers have been in a mad rush over the last decade to produce faster chips at the expense of pretty much anything else. The funny thing is that they've been pushing so hard, they've left a lot of things behind. Take a look at this chart:

You don't have to understand exactly what this means, but the first number relates to "integer" performance, which is important for things like word processing and web browsing and databases, and the second number relates to "floating-point" performance, which is important for games. (Well, and other things too.)

The Pentium M is a modified version of the Pentium III, customised for notebook computers. Since notebooks computers run off batteries, and batteries don't hold much power at all, the Pentium M has been tweaked to provide as much speed as possible while using as little power as possible. The Pentium 4, on the other hand, is designed for speed at the expense of everything else. And what we find is that the 3.2GHz Pentium 4, despite having twice the clock speed of the 1.6GHz Pentium M, is just 25% faster on integer (useful work) and 75% faster on floating point (games).

And - here's the tricky bit, and the cause of Intel's recent and dramatic change in direction - the Pentium 4 uses four times as much power as the Pentium M. So if, instead of putting one Pentium 4 onto a chip, you put four Pentium Ms, it would use the same amount of power and produce the same amount of heat, but it would run up to three times as fast... Overall.

Which is great and wonderfuly if you can use four processors at once. I can, quite happily, and more than that. A word processor can't, not easily, but then word processors already run pretty well. Games, and other graphics-intensive stuff like Photoshop or 3D animation software certainly can, though most games haven't been written to do so. Not yet.* That's dedicated disk; we'll set aside the terabyte or so living in the file server.

Thursday, May 20

MuNu is a blogging community. (It's other things too, but we're talking about the blogs today.) We have personal blogs, public service blogs, group blogs, test blogs, gimmick blogs, joke blogs... We have over one hundred blogs, some with as many as eighty authors; we have over one hundred and fifty authors in total. (It just sort of grew.)

And we kind of like Movable Type. It's not perfect (what is?) but we're used to it.

But. I just added a new blog and a new user, and I've got more people waiting to join, and every Munuvian is free to add guests to their blogs, and that just doesn't work with blog-count and user-count limits.

We'd like to move to MT 3.0. I'm happy to pay for it - and pay more than $69 too - but it needs to be unlimited. Single installation, fine. Non-commerical, okay. But we just can't survive with restrictions on users and blogs.

1
This is where a non-commerical unlimited license would be useful, say perhaps $150 - afterall, why shouldn't all the active author's contribute?

Posted by: Euan at Thursday, May 20 2004 10:20 AM (UVERI)

2
Yeah.
By strange coincidence, an unlimited non-commercial license for Expression Engine is $149. (They even offer a $99 competitive upgrade.)
One of the best features of MT is how easy it is to add a new author or whole new blog. Without that, there's a lot to be said for Wordpress and the other open-source blogging tools.

5
Actually, I don't see what all the uproar is about. If people do not like the pricing for the upgrade, then they don't have to upgrade. I'm sure the plug-in developers won't all just quit coding for the earlier version of MT. Just from reading what others have said about the new versions, I don't see a compelling reason to upgrade. The commenting schemes that are being implemented by the other blog systems seem to be defeating the purpose of comments in the first place. If the customer (reader) is made to do too much work, they won't shop (comment) there. Personally, I'd like to be able to just make the comments and go about my business and not have to jump through even a large hoop. Just my uninformed opinion....

6
MW - true, but there are little things like the MT 2.x templates and stylesheets disappearing from their web site, and not being able to download MT 2.x if you need to re-install...
I need to get back to work on Minx.

9
You should move to Expression Engine. It has not limitation on the author/blogs and it is about 100 times more powerful than MT.
I've been using it since the MT bruhaha and I'm incredibly happy with the support and power of the software.

10
I'm in the same boat. I'm willing to pay a reasonable fee but I have 4 blogs and one of them has 24 authors and is expanding. As of now, they don't offer anything for me at any price. My authors are mostly novices and getting them to learn MT was hard enough. Until I hear a solution from MT for me or until 2.661 crashes and burns, I'm standing pat.
I'm not happy about it, though.

11
I've heard raves about Word Press but it sounds to me like Expression Engine would be the way to go.
I'd be MORE than happy to defray some of these costs (as I'm sure all of the Munuvians whom you rescued from the depths of blogging hell) -- and I'm even willing to help with whatever I can -- migration, uploads, Tylenol, what have you.
Missed ya.
Em

Happiness is relative. Today I don't have a headache; I'm getting over my cold; two of my bosses are out of the office so stress levels are manageable; the new server is finally working... Good enough for me.

Munuvia, and Heather in particular, came under a particularly nasty spam attack today. This one dodges MT Blacklist by generating a new throwaway sub-domain for every spam. And it was persistent. But manually blocking the primary domain stopped it in its tracks. Then I just had to go and delete 400-odd comments. The world blog was saved, thanks once again to Jay Allen and MT Blacklist! Block banned-pics.com now and avoid the rush!

Fedora Core 2 is out - think of it as Red Hat Linux 11. I logged into my home box from work, killed my existing BitTorrent download (Pretty Cure episodes 1-5) and started downloading FC2 instead. It's available as a DVD as well as 4 CDs (8 in all, counting the source CDs as well) but I don't currently have a DVD-ROM drive on my Linux box. I'm not sure quite why I don't, but there it is: A perfect excuse to buy a new 8x DVD burner! (4 CDs, and I bet it still doesn't include Nethack.)

Now I just need to think of an excuse to buy a gigabit switch: I can get a Netgear 5-port switch for $170 now, and my Windows box and my fileserver are already equipped with gigabit cards. Of course, with a mere 2.6GHz Pentium 4, Windows can't really go much above 100mbits anyway, so there's really no point.

On the other hand, it's cheap...

The 1.4 terabytes fileserver at the office has been fixed by the Judicious Application of Money™ - in this case taking the form of a Highpoint RocketRaid 1820 PCI-X 8-port Serial-ATA RAID controller (around A$300). It's a bit fussy, and the drivers don't seem to work with Fedora, but once I'd downgraded to Red Hat 9 everything went smoothly. Nice card, even if it does beep beep beep beep if there's something it doesn't like. Oh, and the ports are numbered 5 - 6 - 7 - 8 - 3 - 4 - 1 - 2, which was a little confusing.

I've been experimenting with rsync for backups - now that we have an extra 1.4 terabytes of space that needs to be filled up - and it works very well. Very well; I'm actually rather impressed. Err... assuming that it's actually working, that is, and it appears to be. It's very quick to backup minor changes to a very large filesystem. Not the best way to back up a live database, but no worse than most of the other ways.

600 gig down, 800 to go. (What do you mean, it's not a contest to see who can fill it up first?!)

3
the b-p jerks hit SR also. Happened to be online when it started, so I went and re-wickered one of the catch-all strings after just picking one and adding only detected 3 of what I could see was a barage.
The re-wicker stopped it, and a rapid purge of over 180 items followed.
I'd like to catch the morons that think this is an effective and useful marketting ploy, or whatever their fucked up line of thinking is, and help stake them out atop a fire ant bed.

Thursday, May 13

For the new server (the 1.4TB server) we bought four Western Digital 200GB SATA drives, and added 4 200GB Western Digital regular-IDE drives we already had.

Two of the four SATA drives were D.O.A. and had to be replaced.

Now the filesystem has gone wonky and the data, as far as I can see, is totally trashed. Fortunately, we have copies of everything on other servers.

I'm running a scan for bad blocks on all the drives. Nothing on the regular IDE drives (so far), and literally hundreds on the SATA drives.

Either we've got a bad batch of drives here (I find it hard to believe that Western Digital is usually this crappy) - or these SATA controllers do not actually work, as such.

Update: If you put two of these SATA controllers in one box, everything appears to work fine until you actually start to use it, at which point the badness sets in. We hadn't planned to do that originally, but then Linux couldn't recognise the SATA controller on the motherboard and...

1
LOL, but with compassion. I feel your pain. I haven't the slightest clue what you're talking about, but I feel your pain.
I did understand the title, however. I could wave my upper appendages erratically, if it would help.

3
You need to make a favicon and add it to the root folder of your web site.
Just thought I would let you know that your site doesn't look good in IE. The right side-bar is half hidden by the centre section. I looked in firefox and it looks fine. The strange thing is that the colors were different in firefox and in IE.

5
Err, how much were the WD 200GB's? I'm looking to upgrade my main box HDD's to free up a spare for the firewall (which is sitting on an ooooooooooold 2GB drive with a history of catastrophic failures).