-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi,
On Friday 22 June 2001 18:56, Brian LaMere wrote:
> couple of follow-up questions....
>> ********************
> I personally try to advocate diskless clients whenever I get a chance.
> There are several reasons for this:
> 1. Administration is much easier if you don't have local data or OS or
> anything that won't be fixed by a simple power cycle. If a problem
> persists through that, you know it's hardware (I guess it could be BIOS
> setup, but I'd consider that hardware).
I agree. OTOH the cluster setup is more complicated IMHO
> ********************
> this is easy enough to automate though...a single shell script that rcp's
> the changed files to all the nodes (for configuration files). Then I was
> thinking of nfs exporting any applications that are needed...simply install
> them into /usr2, and share out /usr2...wouldn't that handle that?
>> ********************
> 2. You do save money on hard disks, but have to spend a little more on
> NICs. What this does mean though, is that you can put a more advanced
> storage system on the server (or network attached storage). Putting
> SCSI Raid or such on the server will increase performance and / or
> reliability (I think Gig E is faster than most IDE drives and I know
> Myrinet is, so you should improve total system performance).
> ********************
>
I partially agree. Your thoughs on memory management?. Swap over Network
would decrease greatly the performance, so in my opinion you are limited to
non-memory-consuming computations
> well, we do use 15k rpm scsi drives, but yeah....ya want to do as few reads
> as possible even then. Just seems with a potential of there being 100 of
> these nodes, I very well may have to boot one every few days...and that
> would be lots of traffic. At that point, is it advisable to have a worldly
> node set aside for doing the boot-up? Hell, I could have everything
> bootstrap off their 100bt ports, then actually work off a gig-ethernet
> port...hmmm...
>> **************************
> 3. In theory, you shouldn't be writing to the local disks in a program
> anyway. This will slow your computations waaaay down. Same thing for
> swapping to disk.
> 4. You only have one copy of data, so not only do you save in total
> storage costs (assuming cost/byte is fixed), but you also only have one
> copy of the data so you don't have to worry about inconsistencies
> between nodes (there are other ways to get the hard drives consistent,
> but they are a bit of a pain). Also, as a sys-admin it is nice to only
> have one place to look if there are problems.
> *************************
> the data would obviously be shared out, yeah. With a possibility of a
> multi-terabyte database that the cluster is querying in a year, there's no
> way I'd put a terabyte on each node..hehe. I mean, IBM can say all they
> want about low-end drives being 400gigs in a year or so, but... point
> being, whether they are diskless or not they'll be pulling in all their
> data locally. Its just the OS that would be local, and perhaps any
> applications I'd want on each node.
>> **************************
> 5. You can call it intuition if you like (although it's based on
> facts),
> but I really think this is the way clusters are going. This is the way
> that big systems like the Cray T3E work. It's a lot simpler for
> programmers (It took me a while to explain how to write to local disks
> vs. server disks). It's also just a lot more elegant, which may sound
> like a cop out, but most good solutions are elegant.
IMHO Cray's are another beasts. Each micro has access to the main memory via
a high-speed bus. Thats not my situation (100Mbps) and probably not the
situation of most beowulfs
> *************************
> Elegancy is certainly not a cop-out in my book. The more elegant something
> is, the better it works...this is almost always true.
>I agree. Having diskless computing nodes focus them as purely computational
nodes, not just some boxen tied to a network
> **********************************
> Having said all that, it is important to note that diskless nodes are
> not for everyone. In fact our cluster is not diskless, and we aren't
> looking at getting diskless nodes any time soon (give me a few years).
> Right now it doesn't meet our needs since we need the local disk and we
> have our disked cluster working fine.
>> Hope this helps a little.
> Jared
> *************************************
Just my $0.02
Cheers
Pedro
>> Any tips are helpful. I'm just sittin here trying to decide which would be
> better for -our- particular application. When is it better for there to be
> disked-clients? Is diskless pretty much something I should obviously do
> considering the fact that the cluster will be quering a huge database
> anyway?
>> Brian LaMere
> Diversa
>> Brian LaMere wrote:
> > why does every guide around talk about diskless clients? I mean...disks
>> are
>> > stinkin cheap nowadays...
> >
> > I have ~$150,000 to make a test cluster (with WAY more if the test
> > cluster shows worth) but the boss-man wants to go with nodes which aren't
> > exactly "commodity" in my book. dual p3-1000 with 1.25Gb ram, 15krpm
> > 18Gb drives. The things cost $8k+ each...tried to explain that 148 $1k
> > machines would
>> way
>> > out perform 16 $8k machines, but...oh well. These boxes take up 1u,
> > which seems to be their main selling point (HP's lp1000r). Fortunately,
> > these boxes are down to $6.5k now in cost (dropped a bit since we bought
> > them a couple months back), but still...
> >
> > on to my point. Getting PVM to see everyone as one happy little family
>> was
>> > easy enough. Got the network guys to isolate the little guys, so that
>> only
>> > the worldly node could see them, since I wasn't happy with opening up
> > everything and simply putting a little all:all in hosts.deny, and having
> > that be all the security I had. But every guide that I've found has been
> > all about diskless nodes for a beowulf. And this isn't really a beowulf
> > with just pvm (and soon lam-mpi and mpich), right? I personally thought
> > that the network nfs/tftp traffic would be horrible if they were all
> > diskless clients...
> >
> > so the real question: I can put gig-e cards in the boxes instead of hard
> > drives...right now they just have 2 100bt enet connections. I'm only
>> using
>> > one of the enet ports at the moment, too. Would I be better with no
>> disks,
>> > and gig-e instead? Some of the concerns I have here: though we're only
> > starting with a hundred gigs or such of data, we'll be at multi-terabyte
> > within a year. To be throwing around data that large, while nfs'ing the
>> OS
>> > filesystems (on the clients) just seems like a lot for the boxes to do.
>> Am
>> > I looking at it wrong? Also, for cost reasons we may be doing our data
> > storage on something as tacky as network attached storage; we were
> > looking at some NetApp boxes, but went with some EMC boxes instead. Note
> > I'm not talking about a symmetrix box or something (I already have one of
> > those housing my oracle data), but instead a EMC product called an
> > "ip4700."
>> Not
>> > all that impressed with it.
> >
> > Just a little genetics research firm, needing some serious horsepower to
> > start running big hammer and blast jobs. The data we have now is just
> > the bare minimum we need to get by, but if we had things like a working
>> beowulf
>> > the scientists upstairs would start making, since they'd be able to use
>> it,
>> > much more data. They hired me on as the unix guy here knowing I don't
>> know
>> > squat about beowulfs, but that I really want to learn :) Got "how to
>> build
>> > a beowulf" <grin> and I've read the manuals for pvm, mpich, lam-mpi, etc,
> > and several other beowulf how-to guides. All are about diskless. Is
> > diskless better? Is it just better because its cheaper? Are there other
> > reasons its better? Would having gig-ethernet in the boxes instead of
>> hard
>> > drives be far better performance-wise?
> >
> > Brian LaMere
> > Diversa
> >
> > _______________________________________________
> > Beowulf mailing list, Beowulf at beowulf.org> > To change your subscription (digest mode or unsubscribe) visit
>>http://www.beowulf.org/mailman/listinfo/beowulf
- --
- -----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org
mQGiBDqcGZsRBADFIahNPLk8suMlS39m8RqatLgX4dO7PU2F5p1oVvkyB7PaLQCv
FREWwfrjGpxAjRnxyZ4TdaFi1oCP495t5R2CdjPZu0EfjsEqosdLXkjDsKl2n4Wo
Afb6BaHMJS5PADEI0QfpZOkB8OruAZja/oGmn5rThyjgCxWHUuK1ArmeGwCg7+9a
owg9wP1RohePHJSDB9d2HYMD/i7z1X4ev+K90LumgJwSWlScJ7MEip5rw4wqGOkK
lF/C2nTYsoX5CVEn/pu7hROL/BWIYtBgkNDaEjsVsyb+4KjQXcZUW5l3ADipWYx2
r9s4sFfeZ9nfhDcG0aNYRcCNkYSZ/WxUkXS8UjVEAEhkFu1BA+6UZmeq3pKtJZTR
+HqKA/9zRmgTon36zt2qe9eiR6DyY0EpGEI0iY+KYX6GC/wxizeHBw0FW1eOEoxF
GjtxdBv/U9vi7Vgav6aY+pr4la5q6jVabe03Y8yGDFeL8jM+lqww1rzpABiGrF+W
qge65zCUjL3jJE5+5yi+KcRyllb1OA7uXQTtsRw+TGq9Dvaaz7QwUGVkcm8gRGlh
eiBKaW1lbmV6IChCLk8uRi5ILikgPHBkaWF6ODhAdGVycmEuZXM+iFYEExECABYF
AjqcGZsECwoEAwMVAwIDFgIBAheAAAoJEJ7ud33hGMZRj20An2Ce4S/vBTuZDxnL
WFBrJRnc3UdaAKDnIPNRbz7r4dh9AuBcpbCE1pQ/SLkBDQQ6nBmqEAQAr7O07Dws
5zAbQvm1hwGthXKCHtIIuWCPdX/XkNG6ZxV/cXgs4LI4oAg3GhttD2JIEk2SoVXE
FOf/wIddIDz70/9mIZavMvpR31LxBFSJk0Up3caOvThM90wMttRi7tg7cf04rrMM
Phy8T5bOIW/q5SMwZffbJXD7bA0/jDLdQ6MAAwYD/1emSwNTzOOmMCZadoEBpKIE
HA35P2/m/SsCI+pQ/OKXKPvvrQKTQqRCcDa5aq31oSiT9M5WQ96BlIGKHRPWGpvm
0822V7M9RF2mYZPIfgKfTSvZpYHzjz+RM7PvBBiBc9l95vy70Sh7SywIF86H80Ag
D0dUIDtGlrSANhXjx4EJiEYEGBECAAYFAjqcGaoACgkQnu53feEYxlHdVACgjVhU
Y8CKf6MYZgQOR9eIDNvTX0AAn3dwbW1HLxEF5OQKJIsngl0BUlYK
=d4S3
- -----END PGP PUBLIC KEY BLOCK-----
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org
iD8DBQE7M85Unu53feEYxlERApXBAJ928Fa0axtRjA6qq1aOH2FCqyqs5wCcCUKU
trEC8zxe8TqY5qnqWQzoJ3I=
=Fqj6
-----END PGP SIGNATURE-----
_______________________________________________
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf