Posted
by
timothyon Monday February 23, 2004 @05:18AM
from the all-the-cool-kids-are-doing-it dept.

dan of the north writes "The NY Times (free reg yyy bbb) is running an article on flash mob computing. More info on the first event in SF on April 3, 2004. The goal is to run Linpack and "build a home-brew computer powerful enough to be added to a list of the world's 500 fastest computers." Minimum requirements are 1.3 GHZ Pentium III/AMD equivalent or better with 256MB of RAM, a 100 Base-T network connection and a CD-ROM - laptops preferred. "After taking a shot at a speed record, the computer will be reorganized to serve as the host of a giant multiplayer video game tournament." Cool... a 2fer!"

You walk into the concert arena, your iPod2 uses zeroconf (rendezvous) to identify itself to the concert host system.

anyone whos' got the 'share my compute cycles' checkbox set on their ipod2 automatically shares their ipod2's cpu cycles, again announced by rendezvous, with the rest of the system.

for the duration of that gig, the machines are all bound together, beowulf-style, to distribute the live recording of the event that is being produced for prosperity during the concert. some 'secrets and surprises' are thrown into the tracks too, just for grins.

at the end of the gig, everyone walks away with their own digital recording of the event, custom, unique, 'branded to the event'.

the whole thing was included in the price of admission, and open to anyone who walks into the concert arena with their boxes turned on...

I can totally see this happening. In fact, if I had the resources, I'd start a company that does just this service for concerts and gigs and such... now would be round about the right time to get into this market, since its infancy-stages...

Will it encourage more people to go to concerts? No. Will they be able to charge people more? No, because the vast majority of concert goers won't have wifi enabled ipods with them, and will resent paying for a service they can't use. Will they be able to bill ipod owners seperately? No, the logistics involved will make it unfeasible. And what if the system doesn't work as advertised? Will they lose the opportunity to sell a live CD later? Yes.

This is an old idea - it's called bootlegging, and with a few notable exceptions, almost all record labels oppose it. Think about it - this idea would be much cheaper and easier to implement if they just offered to send you a CD later and took $5 and a postal address at the T-shirt stall. Plus, they would get revenue from all the non-ipod owners as well, and could fix the parts where the vocalist sings out of tune.

That is certainly the case with Patrick Miller's graduate course in do-it-yourself supercomputing at the University of San Francisco. On April 3, his students plan to assemble the first "flash mob supercomputer" in the school gym.

While brainstorming about how to build a home-brew computer powerful enough to be added to a list of the world's 500 fastest computers, Mr. Miller and his students, along with Gregory D. Benson, an associate professor of computer science, came up with the idea of an electronic barn-raising. They decided to build on the concept of flash mobs, the sudden Internet-organized gatherings with no particular purpose that became an unlikely fad last summer.

Last week, the class put out a call for about 1,200 volunteers to bring their computers to the Koret Gym here for a day and plug them into a shared high-speed network.

"This is what happens when crazy ideas catch fire and people say, `Wait, there is nothing to stop this,' " said Mr. Miller, who is a lecturer at the university and a computer scientist at the Center for Applied Scientific Computing at the Lawrence Livermore National Laboratory.

There are already many Internet-connected virtual supercomputers, like the SETI@home project, which uses the spare computing cycles on the personal computers of volunteers to hunt for signs of alien civilizations. Several universities have shown that it is possible to hook hundreds of off-the-shelf personal computers together to create supercomputers. But until now no one has tried to build an instant supercomputer in one place.

"It struck me as being something of a 60's idea," said Dennis Allison, a founder of Dr. Dobbs, a Silicon Valley magazine for computer programmers. "This could easily be an idea from one of William Gibson's science-fiction novels, where everyone gathers in Grand Central station to save the world by plugging their machines into the Net."

Before stumbling onto the idea of the volunteer project, the class considered a variety of ways to make a cheap supercomputer, including buying many Microsoft Xbox game machines. However, the students would have needed to install the free Linux operating system on the machines to tie them together, and Microsoft has recently made that more difficult.

John Witchel, the graduate student who had the original idea of building a volunteer supercomputer, says he thinks flash mob computing will make it possible for high school students and community groups to harness computer power now available only to large corporations or government laboratories.

"We're trying to democratize supercomputing," Mr. Witchel said.

The group has high hopes for its gym machine. It plans to run a speed benchmark program known as Linpack. The group estimates that to make the next Top 500 list, scheduled to be released in June, the machine will need to reach a speed of about 550 gigaflops, or billions of mathematical operations per second. The No. 1 spot on the list is held by the Earth Simulator in Japan, which can run at more than 35 teraflops, or 35,000 gigaflops.

Jack Dongarra, a University of Tennessee computer scientist who helps maintain the Top 500 list, says the students have a shot at making the list, but it will not be easy.

"It could be that electrical power will be an issue," he said, adding that the slowest computer will limit the speed of the entire supercomputer. To make certain that they have enough speed, the students are asking that volunteers bring only computers with at least a 1.3-gigahertz Pentium or AMD processor and 256 megabytes of memory, requirements that most recent home machines will meet. Laptops are preferred because they use less power than desktop computers.

When all the machines are plugged together via donated high-speed networking switches, the students will be able to tack

The article clearly notes that the this will be used to estimate the Airspeed Velocity of an Unladen Swallow. If there's enough time the organizers said they would try for both the African and European Swallow.

You've either never been to a significantly large LAN party or are incredibly lucky. Getting x,000 randomly selected laptops to even all communicate together properly for the benchmark will be a major undertaking, nevermind doing any useful work in the amount of time allotted. The planners give the impression of being quite organized with their pre-made Knoppix disks but I assure you there will be something to gum up the works. This leads to a whole new discussion of why can't PC's be plug-n-go appliances after 20+ years, but nevermind that now...

This doesn't sound like such a bad idea. seems like a lot of things could be done this way much faster and more efficiently. if this works we should look into other applications for the Flash Mob, like a rocket-building day for the X-Prize, or a random code-swap where a bunch of us get together and hand eachother a blank disc with the source code to something nifty on it to play with.

Seriously, I'm worried that the very smallest mishap will bring this crashing down. If this works, it'd be the greatest thing ever, but if it doesn't, what a spectacular failure. It'd be interesting to count how many power cords are tripped over in the process.

What's amusing is that people are encouraged to bring laptops, and are then expected to play games that way...

What's amusing is that people are encouraged to bring laptops, and are then expected to play games that way...

Maybe you're trolling, but laptops can be perfectly serviceable game machines. They may not be able to match an absolute top-of-the-line desktop, but with DDR3200, 7200 RPM HD's, and 128MB graphics cards, they can be pretty competitive.

On a related note, I've never seen anyone on/. mention iBuyPower [ibuypower.com]. Their prices are $1000 less than Alienware, and probably a few hundred less than a Sager or Clevo. They really are in a (price) class of their own.

The next thing you know, SkyNet will be born from this cluster! Especially considering this would take place on the Governator's state.;)
Better get those EMPs ready though, I'm expecting robots from both Terminator and Matrix to come to life after a few days.;)

As a USF Student, I will be there with my laptop...All I have to say is GO DONS... The computer science department already has a cluster called the "keck cluster". Basically 64 nodes of dual P3 at 1GHz, with 1 Gig ECC Ram. There is talk about throwing the keck cluster into the flash mob cluster, but the biggest hurdle is appearntly laying the lines. Harney Science center is about 200 hundred yards from the gym where this is going to happen.
And just FYI, they wanted it to be done on the 1st of april, but that didn't work out for some reason.

Maybe somebody should point out that is not the first time somebody has done such a thing... back in 1998 there was a quite similar event at the University of Paderborn where 512 normal home PCs brought by people were connected for one night (the event was even broadcasted live on German TV). I have to admit that the "flash mob" element here is more predominant (back then people knew about this two weeks in advance), but it's definitely not the first attempt to create a spontaneous supercomputer with home machines. The cluster even made it into the Top250 IIRC.:)

Key difference between cluster computing and grid computing: connection between the nodes. Grid computing works best with relatively low amounts of data on which much individual computation has to be performed. Clusters can handle bigger data and more interrelated computation.

From the FlashMob FAQ:"How do I setup my own flashmob supercomputer?
First and foremost, if you can come to FlashMob I -- there's no substitute for first hand experience. Otherwise, start here and get some experience running a one node flashmob. Then run two. Then run ten. Then take over the world."

We're gonna to do it on your computer, then two computers, then ten computers, then your neighborhood. And then we're gonna do it at USF, then California, and then we'll take over the world! YEEEEEEAAARRRRGGHHH!!!

guys from the CLOWN '98 [http://www.tlachmann.de/linux-cluster/] already tried this (even it was not the main goal). it was a temporary cluster for only one night, but to get into top500 you have to build a durable cluster.

I have an idea as how to make Wifi hotspots economical. Imagine the Matrix meets Slashdot - in our wifi hotspot, imagine a network that sucks the living cycles from a beowulf cluster of you!

Here is the deal... to use a hotspot you have to download a package that connects your computer to the local "grid". In exchange for network access the grid gets your spare CPU cycles. The best hotspots could leverage the power of hundreds of notebooks, and then resell this on the market as a computing resource commodity, for multiplayer games, data crunching, whatever.

Though... I'm running a high fever and this is perhaps the fruit of a deranged mind.

The next logical step I think would be for a university to provide free internet in dorm rooms, as long as you leave your system on and run a distributed computing client for them.
The student saves $$ without any noticable problems on their side, the university gets free computing time, seems like a win win situation.

I don't think it will work. Reliability, bandwidth and latency tend to be the biggest problems with purpose built clusters. I imagine a setup like the one which you describe is going to be much worse in these aspects. Not to mention that most simulation software is written for homogeneous clusters, i.e. all the computers in the clusters should have an identical software environment, and the slowest node limits the speed of the entire calculation.

A similar but slightly less pie-in-the-sky thing would be to use the lab computers for batch processing during the night. Lab computers tend to be centrally administered and perhaps identical computers could be grouped together so that CPU cycles aren't wasted to the extent that they would be in a homogeneous environment.

If only that could assure a decent connection. Where I'm at, the connection isn't even worth being a freebie. It's slower than dialup in a lot of cases, and completely unreliable (it went down at least 10 times just this past weekend). Then there's the fact that the support office is a complete joke. A friend of mine took their computer over there because the network adapter was being a pain in the ass. They didn't know what to do with it. The sad thing is, it's an adapter my friend bought at the campus boo

I really think 1200 is overkill for this. Take a look at the Top500 list and see how badly all the Gigabit Ethernet systems scale: most of them have worse than 50% efficiency and that's with only 1/4 the number of nodes. Now cut the interconnect bandwidth by a factor of TEN, cause Apple is pretty much the only company putting Gig-E standard in their (pro) computers, and it seems to me that hundreds of people are going to be sent packing because adding them to the cluster would actually make it SLOWER.

New Thinkpads (at least my T40) also are shipping GigE. I'm fortunate to have both a poweredge at my desk and a T40 to wander about with... if only I could do something with those GigE interfaces other than enjoy their very fun autodetection (no crossover cables, ever!)

SETI@HOME isn't on the Top500 list, because it's not running Linpack, but according to its stats page, it's been running at about 63 TeraFLOPS today, which is comfortably #1 on the list. So this should be fun...

Having done a fair bit of linpack benchmarking on 'real' clusters I have to say I think their chances are very slim. The interconnect makes a huge difference particularly when you have so little RAM per node. I doubt they will have a nonblocking switch architecture which makes a significant difference in Linpack (even setting aside it not being gigabit.) Also, MPI applications (Linpack included) often run into bottlenecks with wait conditions, some of the slower nodes will probably end up choking the entire cluster. A few problem laptops with bad RAM modules, and they can spend more time than they have pulling their hair out troubleshooting.

That being said, it doesn't seem like all that serious an enterprise. Good luck to them, and if they have fun, hey all the better.:)

Given how long this will take to set up, battery power isn't a viable option. Still, using a laptop is a pretty good idea. If you compare a laptop drawing 65 watts to a desktop drawing 300 at full CPU utilization, with a knoppix CD spinning at full speed, plus monitor power, you see that they gain nearly an order of magnitude in energy efficiency, though this is probably offset a bit by the lower clock speed on the laptop processors.

Now, let's generously assume that each laptop is drawing half an amp at 110 volts. At 1200 laptops, that's 600 amps. The circuit breakers in my house trip at 15 amps, but I'll generously assume this facility has 50 amp wall circuits. That would still require 12 entire circuits, plus a safety factor, nevermind all my generous back-of-envelope assumptions.

Okay, so assuming they've got a lot of extension cords, now we just have to deal with space. Let's assume, again, generously, that each person + computer + associated infrastructure needs only one square meter of floor space. This makes the space requirement equivalent to a 30m x 40m area, or about two World Cup soccer fields. I hope they've got one hell of a big gym.

Heat is, by comparison, a relatively minor issue. If the facility can handle a crowd that large, adding their low-power laptops is minor. People tend to dissipate about 100 watts anyway, so the laptops won't be the most significant source of difficulty.

It sounds like a very daunting task they have ahead of them. I hope they've already gotten these problems figured out, because this project sounds really cool.

I don't know what the maximum power draw of a fast notebook at 100% CPU is, but I do know that a desktop is nothing like 300W. More like 90-120W -- sans monitor of course, since a cluster would never have one monitor per node. And that's a real full-power CPU, not the throttled down "mobile" version. You simply cannot put a 70W CPU in a notebook, it would melt. So notebook CPUs have to be slower even on AC power than standard CPUs. Notebooks are not going to give a very big computron/watt advantage, if

1. Heat: If they expect to have 1200 people sitting in the bleachers with 1300+ MHz laptops running at 100% for the duration of a real benchmark, they'll have to expect a lot of burned laps.

2. Theft: If they have a designated hookup area for the computers (much more logistically feasible, ethernet-wise at least) they'll have a hard time getting all 1200 computers successfully reunited with 1200 owners. It'd be very easy to grab the wrong one "accidentally". Given t

The 300W is more than just the CPU. There's a lot of other stuff that draws power in a computer, like the power supply, the memory, the disk, etc. The nice thing about underclocking is that power draw is proportional to the cube of the clock frequency (this becomes more of a quadratic relationship as you get away from the peak performance of the chip), so you get huge power savings from modest underclocking. You also get a better cpu/memory clock ratio, so the pain is still less.

Now, let's generously assume that each laptop is drawing half an amp at 110 volts. At 1200 laptops, that's 600 amps. The circuit breakers in my house trip at 15 amps, but I'll generously assume this facility has 50 amp wall circuits. That would still require 12 entire circuits, plus a safety factor, nevermind all my generous back-of-envelope assumptions.

600 amps is nothing for a place like this. I can get an 800 amp feed to my house, according to the nice folks at my local power company (and I'll be upgrading to that as part of my rewiring plan). 12 circuits is no big deal either. Think of the power requirements for holding concerts or any type of show, or just for lighting the place.

A place like that also probably has a house electrician on staff who knows the boards and how to get x amps from point A to point B. I doubt power is going to be the main issue holding them back on this.

A javascriptlet there will allow you to generate a totally random login for viewing the article. Every Slashdotter which accesses the article should create a new random login in turn, filling their database with useless random login id's that are only used once and then forgotten about.

The goal is to run Linpack and "build a home-brew computer powerful enough to be added to a list of the world's 500 fastest computers."

Yes, but you require a minimum of a 100 Base-T connection. You want to create one of the world's top 100 supercomputers using Ethernet? Good luck in beating that latency, guys....next time, see if you can get a flash mob of infiniband vendors to come along for the ride.

Exactly, speaking as someone who has run CFD code on smallish clusters, 100MB ethernet falls flat on it's face at about 20 machines, and those were dual 500MHz machines - it will be worse with faster ones! (to preempt some silly comments, CFD code and linpack have a LOT in common)

And that was using specially tuned low latency ethernet drivers and TCP stack under linux.These guys have very very little chance of doing anything useful at all - which is a bit of a pity, but perhaps if they did just a little research first..

I wonder if they even have network switches that will efficiently route 1200 nodes.. let alone a decent plan to interconnect them.

The first step would be to use 1Gbit or faster concentration to some very smart switches to at least cut down the network blockage a little.. It won't help with the terrible latency, but will give them a little headroom at some vector lengths.

They will also suffer terribly from the differing speeds of nodes - I've yet to see a solution for linpack that distributes efficiently over a wide speed range of machines.

Of course, I bet in the end they just come up with a great SETI score, or something similar - something that would actually scale at all on a cluster like this.

Minimum requirements are 1.3 GHZ Pentium III/AMD equivalent or better with 256MB of RAM
Perhaps I'm missing some fundamental requirement of cluster comptuing, but why wouldn't macs work? I'm sure a 1.25 ghz G4 could hold its own with the above mentioned. It can also run linux. College campuses seem to be a hotbed of mac users, so it seems that they would want to tap this. Does clustering require that all nodes be of the same architecture?

I'm not sure the SOFTWARE aspect of this "Super Computer" but, uh... maybe the programmers are coding this in a Microsoft language...

Mentioned before is SETI@home (software distributed amongst many clients) and another (I'm too lazy to scroll back up and read). This "Psuedo-Supercomputer" aspect is mearly a distributed software application which shares data over a network. All hosts must run a program to "Link" them together in such a manner that they distribute the computing. Therefore, the pl

apparently I had no idea what I was talking about... it appears that they are booting from a linux cd that will be provided... so to answer your question... macs would probably work providing they are new enough to run linux...

Well. The Knoppix linux distro, which is to be used, is prepared for x86 architecture. Sorry, guys. Maybe it's time to start thinking about porting this distro to Mac CPUs? It shouldn't be that hard...

Interesting term "flash mob". The first time I encountered it was in the writing of sci-fi author, Larry Niven. In some of his series personal teleportation becomes ubiquitous giving rise to the 'flash mob'.

When a news broadcast reports a certain kind of story (riot, fire, etc) people start to teleport into watch the fun. The news reports the growing mob and before long it reaches critical mass and turns into a real riot as people take advantage by teleporting in and doing a quick bit of looting.

I'm not sure if Larry originated the term though ? Anyone know an ealier source ? Is it a 'real' phenomenon ?

Flash Mobs DO exist these days, but generally aren't disaster tourists. Yet, anyways. Modern Flash Mobs consist of large numbers of peoples gathering somewhere, doing something silly for about 5 minutes and then leaving again, just as quick. An example I heard of some time ago was at a Toys R Us store ages ago with a huge mock-up dinosaur. A Flash Mob gathered before the dinosaur and worshipped it for 5 minutes. And I'm not talking about quiet, solemn worship, here. After 5 minutes the crowd wa

My first thought was "Neat." My second thought was "Who in their right mind would connect their computer to a network FULL of strangers? What sort of viruses and trojans would they pick up?" Then I remembered that I am connected to the internet as well as everyone reading this...

Sure it's a cool stunt, but aside from running the Linpack benchmark, what will this pile 'o' pcs do? It will, of course, do nothing, for it will only exist about a day.

I've seen this sort of thing happen before: people devoting energy and money to what amounts to a technical fetish. The end state is a world where people like Robert G. Brown build themselves home beowulf clusters with no discernible purpose. (RGB: you're a nice guy and all, but I find it hard to believe that you need all that horsepower for personal use).

I'd rather see an article about broadband users organising themselves into a GLOBUS grid. For that matter, I'd like to see a comprehensive system for bug tracking MPICH (I've seen some weird bugs there). There's lots of things I'd like to see written about or developed. Tomorrows 'infinity + 1' Supercluster ain't it.

From flashmobcomputing.org: "Today, supercomputing is controlled largely by governmental organizations, academic research institutions, animation studios, and recently human genome companies. This means that the problems that get solved by supercomputers are narrow in scope and tightly controlled. We want to change that."

This is so stupid.

"supercomputing is controlled" - what does that mean? Nothing!

"narrow in scope" - What possible problem can one want to solve on Beowulf-type cluster? The only one

...that you don't hear about: Large collections of ping zombies, and large collections of SMTP spam relays under hostile control (I can't think of a better term than 'hostile'). Come on, how many distributed supercomputers out there have tens of thousands to millions of nodes?

This sounds like a fun event, but I'd be very surprised if they can make this work. Sure, if you get enough computers together, the "theoretical peak" (basically the aggregate FLOP rate of all the computers) can be really high, but for a problem with dependencies like the LINPACK benchmark, communication latency is what is going to limit the runtime when you try to hook a thousand or so laptop computers together using only fast ethernet. I think that even Gigabit ethernet would scale poorly in such a setting (decent bandwidth, but startup time for a communication is slow compared to something like Myrinet or Infiniband).

Maybe it's possible, but I think it would require a pretty creative network topology and some pretty clever re-structuring of the linpack benchmark (which is allowed by the Top 500 list rules, BTW).

How is this a flash mob if it's advertised weeks ahead and isn't going to just show up and then disperse quickly? This is just a planned event with no RSVP list, that doesn't make it a flash mob, they're just jumping on that currently-hip (not for long let's hope) name to get attention and feel cool. I'm surprised they didn't actually say "it's a flash mob of metrosexuals!" to really get attention from lazy people who love idiotic hip slang.

...like a KNOPPIX ISO image that is specifically meant for setting up a distributed computer on a local LAN. How about it? I can think of a lot of places that could use the power of distributed computing that don't necessarily have all the knowhow of how to set it up - let's take the most obvious : public schools. Something like a Flash Mob Computing KNOPPIX ISO downloadable would be like a gift from God.