Finally, after two weekends and hours of work, I get to do what I started out trying
to do - building a six drive RAID 5 array out of terabyte hard drives. Cartman's old
5U case was all cleared out, I had all the components, now all I had to do was assemble
the beast. I had a little problem with the Adaptec
3805 controller.

The 3805 is actually an SAS controller, using the mini-SAS plugs
that handle four drives each. On the web site the specification says that the board
comes with a pair of mini-SAS to SATA cables, but there were no such cables in my
box. Turns out I had ordered the OEM version of the board (the only one available),
and it had no cables in it - which makes sense, its an OEM board, the OEM is always
going to want to do something unique with the board.

Fine, I'll order my own cables. But NOBODY has stock on mini-SAS cables. I flip out
at the supplier, and he calls Adaptec, and they offer to give me a pair of cables
for free (which was mighty nice of them), if I'll pay the shipping. Totally worth
it, I had ordered the wrong product and they were willing to fix it. A FedEx overnight
shipment later, I had cables.

There's so much room in the 5U case that things came together rather quickly. The
motherboard dropped in without a hitch, as did the drive array caddy. Then came the
tricky bit...

Like
Cartman, Butters has a separate pair of mirrored boot drives, although in
this case the drives are 7200rpm SATA II drives, rather than the Ultra-160 SCSI
drives of Cartman.

In the 5U case, the boot drives hang from the card retaining bar... and the first
hitch of the build occurs. In a test hanging (shown to the right), the pair of drives
hit the CPU fans. This is bad.

When a situation like this arises, first you curse. Then the full reality of the situation
hits - all the work you've done for the past few days may have been for naught, this
machine won't fit into this case.

I ran into the same
issue with Cartman during his rebuild, I had to modify the cooling blocks to use
lower-profile fans to avoid conflicting with the hanging hard drives. But I didn't have
that option this time... no handy low-profile fans, no alternative cooling blocks.
I needed a different solution.

And
here's the solution - move the drives. It's not like the new machine is full
of cards anyway, it has exactly one, the Adaptec 3805 raid controller. And that card
is low-profile anyway.

So I removed all the card holders from the bar and moved the mounting bracket so that
the drives would hang away from the CPU fans. Problem solved.

That was really the only hitch in the assembly of Butters, and it only took me a few
minutes to solve it. I like this new drive position better, it puts the drives right
inline with the main fan, so there'll be plenty of cooling air coming over those drives.

A little more fussing with wiring and I was on my way with a successful boot
of the new motherboard...

Notice
that I plugged one of the 1TB drives into the machine as well, getting ready for the
transfer of all that data back onto a shiny new 5TB array.

Ah, if only it was that easy. First I had to get a server install done. Which you
think would be easy - a brand new motherboard, it should be no problem to get things
up and going with Windows Server 2003, right?

Wrong.

Since I was planning to use this machine to run virtual machines, of course I wanted
a 64 bit operating system on it - there's 16GB of RAM in there, how else would I address
it all?

So I installed Windows Server 2003 SP2 64 bit edition. And the installation went cleanly,
but didn't recognize the pair of built-in gigabit NICs. I wasn't all that surprised,
after all, brand new motherboard, I'd need to install the drivers separately. Now
if only I could find them.

On the Tyan web site you can see all sorts of drivers
for the S2927, including drivers for Windows 2003 Server 64 bit, so you'd think
there would be NIC drivers there. In fact, under the heading "Driver Packs" there
is a pack for Windows 2003 Server 64 bit which SAYS it has LAN/NIC drivers. However,
if you actually download it, there's no NIC drivers in there. In fact, if you open
up the zip file, the README doc lists everything in the driver pack and it does NOT
include the NIC drivers.

I tried installing it anyway, but to no avail - the NICs were still unrecognized.

However, the Adaptec software worked great AND the 5TB array was able to be built.
But it was going to take more than 24 hours to prep itself, so it was worth tinkering
with other configurations before settling for this.

So I headed over to the nVidia site... perhaps the reference drivers for the nVidia
chipset would handle the NICs better. The chipset on this motherboard is the nVidia
nForce Professional 3600 series. And lo and behold, there ARE reference
drivers for Windows 2003 Server 64 bit. But they TOO could not recognize the NICs.

I even tried the prerelease
tool on the download page to detect what drivers to use, and it recommended the
Vista drivers! Figuring it couldn't be any worse, I tried them too... and this time
the NICs were recognized, but were not functioning.

So now I'm afraid - afraid that my motherboard is defective. But now that I have nothing
to lose I thought "what the heck, let's try Windows 2008 Server!" I had Release Candidate
0 handy, it was worth a shot.

Windows 2008 Server RC0 is a massive 2.5GB, I had to make a DVD for the install. But
it installed flawlessly and recognized the motherboard, including the NICs. I was
fully operational. And Windows 2008 Server is beautiful... but its a release candidate!

So now my motherboard was working perfectly, I installed the Adaptec RAID controller
software. It installed, recognized the controller AND the drives. For the first time
I had everything working, admittedly on a release candidate. How could I resist? I
configured the 5TB array and let it rip.

The build ran overnight and finished perfectly. I had a 5TB drive array!

I shutdown Butters, closed it up and stuck it in the rack.

Powered it up again, but when it booted, there was no drive array! I rebooted again,
still no array. What was going on? Pulled Butters back out of the rack, opened it
up, booted it again... still no array.

I went into the 3805 BIOS to configure the array and it didn't show up until I selected
"Refresh Array." Then it showed the complete array, in perfect condition!

Baffled, I exited the BIOS settings which caused a reboot... and the array vanished
again. This time when I finished booting into Windows, I opened up the Adaptec configuration
manager... it showed a failed controller and failed drivers. I selected "Refresh Array",
and it still showed as everything failed - but Windows suddenly found the array! The
drive letter popped up and everything acted fine.

Oddly enough, I was a bit suspicious.

So I started loading data onto the array. I wasn't going to erase any backups, so
I waited for it to fail.

Loading went much faster than backing up, since the drive was plugged directly into
the machine. Within a few hours, I had everything reloaded.

I was still suspicious.

I configured the file shares and got both the music and television archives up and
running. They worked perfectly.

Now I really had a problem - I was running a release candidate OS, the configuration
software says the array has failed (although the BIOS says its fine, once you refresh),
but Windows itself is perfectly happy with it. And my family was happy to have the
music and video back online. I couldn't very well take it back down. As long as it
didn't reboot, the array seemed to stay up. Scary.

I sent a tech support request to Tyan, hopefully they'll have something useful to
say. I really ought to go back to Windows 2003 Server 64 bit, but only if I can get
the NICs to work.

Storage Upgrade Stage 3 - Building Buttershttp://www.campbellassociates.ca/blog/PermaLink,guid,a3e2d050-47f0-447b-a44a-20ea5b2a1a08.aspxhttp://www.campbellassociates.ca/blog/PermaLink,guid,a3e2d050-47f0-447b-a44a-20ea5b2a1a08.aspx
Mon, 15 Oct 2007 00:33:23 GMT<p>
Finally, after two weekends and hours of work, I get to do what I started out trying
to do - building a six drive RAID 5 array out of terabyte hard drives. Cartman's old
5U case was all cleared out, I had all the components, now all I had to do was assemble
the beast. I had a little problem with the <a href="http://www.adaptec.com/en-US/products/sata_tech/value/SAS-3805">Adaptec
3805 controller</a>.
</p>
<p>
The 3805 is&nbsp;actually an SAS controller, using the <a href="http://www.techcable.com/HTML/MINI-SAS.htm">mini-SAS</a> plugs
that handle four drives each. On the web site the specification says that the board
comes with a pair of mini-SAS to SATA cables, but there were no such cables in my
box. Turns out I had ordered the OEM version of the board (the only one available),
and it had no cables in it - which makes sense, its an OEM board, the OEM is always
going to want to do something unique with the board.
</p>
<p>
Fine, I'll order my own cables. But NOBODY has stock on mini-SAS cables. I flip out
at the supplier, and he calls Adaptec, and they offer to give me a pair of cables
for free (which was mighty nice of them), if I'll pay the shipping. Totally worth
it, I had ordered the wrong product and they were willing to fix it. A FedEx overnight
shipment later, I had cables.
</p>
<p>
There's so much room in the 5U case that things came together rather quickly. The
motherboard dropped in without a hitch, as did the drive array caddy. Then came the
tricky bit...
</p>
<p>
<a href="http://www.campbellassociates.ca/blog/content/binary/WindowsLiveWriter/StorageUpgradeStage3BuildingButters_22F0/IMG_7868_small.jpg" atomicselection="true"><img style="border-top-width: 0px; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" height="350" alt="The drives don't fit!" src="http://www.campbellassociates.ca/blog/content/binary/WindowsLiveWriter/StorageUpgradeStage3BuildingButters_22F0/IMG_7868_small_thumb.jpg" width="526" align="left" border="0"></a>Like
Cartman,&nbsp;Butters has a separate pair of&nbsp;mirrored boot drives, although in
this case the drives are&nbsp;7200rpm SATA II drives, rather than the Ultra-160 SCSI
drives of Cartman.
</p>
<p>
In the 5U case, the boot drives hang from the card retaining bar... and the first
hitch of the build occurs. In a test hanging (shown to the right), the pair of drives
hit the&nbsp;CPU fans. This is bad.
</p>
<p>
When a situation like this arises, first you curse. Then the full reality of the situation
hits - all the work you've done for the past few days may have been for naught, this
machine won't fit into this case.
</p>
<p>
I ran into the <a href="http://www.campbellassociates.ca/blog/PermaLink,guid,32cc0c54-869b-415b-b2a0-531cd40d3eb4.aspx">same
issue</a> with Cartman during his rebuild, I had to modify the cooling blocks to use
lower-profile fans to avoid conflicting with the hanging&nbsp;hard drives. But I didn't&nbsp;have
that option this time... no handy low-profile fans, no alternative cooling blocks.
I needed a different solution.
</p>
<p>
<a href="http://www.campbellassociates.ca/blog/content/binary/WindowsLiveWriter/StorageUpgradeStage3BuildingButters_22F0/IMG_7869_small.jpg" atomicselection="true"><img style="border-top-width: 0px; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" height="350" alt="Solution - move the drives." src="http://www.campbellassociates.ca/blog/content/binary/WindowsLiveWriter/StorageUpgradeStage3BuildingButters_22F0/IMG_7869_small_thumb.jpg" width="526" align="left" border="0"></a>And
here's the solution - move the drives. It's not like the&nbsp;new machine is full
of cards anyway, it has exactly one, the Adaptec 3805 raid controller. And that card
is low-profile anyway.
</p>
<p>
So I removed all the card holders from the bar and moved the mounting bracket so that
the drives would hang away from the CPU fans. Problem solved.&nbsp;
</p>
<p>
That was really the only hitch in the assembly of Butters, and it only took me a few
minutes to solve it. I like this new drive position better, it puts the drives right
inline with the main fan, so there'll be plenty of cooling air coming over those drives.
</p>
<p>
&nbsp;A little more fussing with wiring and I was on my way with a successful boot
of the new motherboard...
</p>
<p>
<a href="http://www.campbellassociates.ca/blog/content/binary/WindowsLiveWriter/StorageUpgradeStage3BuildingButters_22F0/IMG_7870_small.jpg" atomicselection="true"><img style="border-top-width: 0px; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" height="350" alt="IMG_7870_small" src="http://www.campbellassociates.ca/blog/content/binary/WindowsLiveWriter/StorageUpgradeStage3BuildingButters_22F0/IMG_7870_small_thumb.jpg" width="526" align="left" border="0"></a> Notice
that I plugged one of the 1TB drives into the machine as well, getting ready for the
transfer of all that data back onto a shiny new 5TB array.
</p>
<p>
Ah, if only it was that easy. First I had to get a server install done. Which you
think would be easy - a brand new motherboard, it should be no problem to get things
up and going with Windows Server 2003, right?
</p>
<p>
Wrong.
</p>
<p>
Since I was planning to use this machine to run virtual machines, of course I wanted
a 64 bit operating system on it - there's 16GB of RAM in there, how else would I address
it all?
</p>
<p>
So I installed Windows Server 2003 SP2 64 bit edition. And the installation went cleanly,
but didn't recognize the pair of built-in gigabit NICs. I wasn't all that surprised,
after all, brand new motherboard, I'd need to install the drivers separately. Now
if only I could find them.
</p>
<p>
On the Tyan web site you can see all sorts of <a href="http://www.tyan.com/support_download_drivers.aspx?model=S.S2927">drivers
for the S2927</a>, including drivers for Windows 2003 Server 64 bit, so you'd think
there would be NIC drivers there. In fact, under the heading "Driver Packs" there
is a pack for Windows 2003 Server 64 bit which SAYS it has LAN/NIC drivers. However,
if you actually download it, there's no NIC drivers in there. In fact, if you open
up the zip file, the README doc lists everything in the driver pack and it does NOT
include the NIC drivers.
</p>
<p>
I tried installing it anyway, but to no avail - the NICs were still unrecognized.
</p>
<p>
However, the Adaptec software worked great AND the 5TB array was able to be built.
But it was going to take more than 24 hours to prep itself, so it was worth tinkering
with other configurations before settling for this.
</p>
<p>
So I headed over to the nVidia site... perhaps the reference drivers for the nVidia
chipset would handle the NICs better. The chipset on this motherboard is the nVidia
nForce Professional 3600 series. And lo and behold, there ARE <a href="http://www.nvidia.com/object/nforce_pro3000_winserver2003_x64_9.24.html">reference
drivers</a> for Windows 2003 Server 64 bit. But they TOO could not recognize the NICs.
</p>
<p>
I even tried the <a href="http://www.nvidia.com/Download/index.aspx?lang=en-us">prerelease
tool</a> on the download page to detect what drivers to use, and it recommended the
Vista drivers! Figuring it couldn't be any worse, I tried them too... and this time
the NICs were recognized, but were not functioning.
</p>
<p>
So now I'm afraid - afraid that my motherboard is defective. But now that I have nothing
to lose I thought "what the heck, let's try Windows 2008 Server!" I had Release Candidate
0 handy, it was worth a shot.
</p>
<p>
Windows 2008 Server RC0 is a massive 2.5GB, I had to make a DVD for the install. But
it installed flawlessly and recognized the motherboard, including the NICs. I was
fully operational. And Windows 2008 Server is beautiful... but its a release candidate!
</p>
<p>
So now my motherboard was working perfectly, I installed the Adaptec RAID controller
software. It installed, recognized the controller AND the drives. For the first time
I had everything working, admittedly on a release candidate. How could I resist? I
configured the 5TB array and let it rip.
</p>
<p>
The build ran overnight and finished perfectly. I had a 5TB drive array!
</p>
<p>
I shutdown Butters, closed it up and stuck it in the rack.
</p>
<p>
Powered it up again, but when it booted, there was no drive array! I rebooted again,
still no array. What was going on? Pulled Butters back out of the rack, opened it
up, booted it again... still no array.
</p>
<p>
I went into the 3805 BIOS to configure the array and it didn't show up until I selected
"Refresh Array." Then it showed the complete array, in perfect condition!
</p>
<p>
Baffled, I exited the BIOS settings which caused a reboot... and the array vanished
again. This time when I finished booting into Windows, I opened up the Adaptec configuration
manager... it showed a failed controller and failed drivers. I selected "Refresh Array",
and it still showed as everything failed - but Windows suddenly found the array! The
drive letter popped up and everything acted fine.
</p>
<p>
Oddly enough, I was a bit suspicious.
</p>
<p>
So I started loading data onto the array. I wasn't going to erase any backups, so
I waited for it to fail.
</p>
<p>
Loading went much faster than backing up, since the drive was plugged directly into
the machine. Within a few hours, I had everything reloaded.
</p>
<p>
I was still suspicious.
</p>
<p>
I configured the file shares and got both the music and television archives up and
running. They worked perfectly.
</p>
<p>
Now I really had a problem - I was running a release candidate OS, the configuration
software says the array has failed (although the BIOS says its fine, once you refresh),
but Windows itself is perfectly happy with it. And my family was happy to have the
music and video back online. I couldn't very well take it back down. As long as it
didn't reboot, the array seemed to stay up. Scary.
</p>
<p>
I sent a tech support request to Tyan, hopefully they'll have something useful to
say. I really ought to go back to Windows 2003 Server 64 bit, but only if I can get
the NICs to work.
</p>
<img width="0" height="0" src="http://www.campbellassociates.ca/blog/aggbug.ashx?id=a3e2d050-47f0-447b-a44a-20ea5b2a1a08" />http://www.campbellassociates.ca/blog/CommentView,guid,a3e2d050-47f0-447b-a44a-20ea5b2a1a08.aspxRackmountingToysVirtualizationWindows Server 2008http://www.campbellassociates.ca/blog/Trackback.aspx?guid=8bf006bc-d3d3-48d9-baa9-738350801c55http://www.campbellassociates.ca/blog/pingback.aspxhttp://www.campbellassociates.ca/blog/PermaLink,guid,8bf006bc-d3d3-48d9-baa9-738350801c55.aspxhttp://www.campbellassociates.ca/blog/CommentView,guid,8bf006bc-d3d3-48d9-baa9-738350801c55.aspxhttp://www.campbellassociates.ca/blog/SyndicationService.asmx/GetEntryCommentsRss?guid=8bf006bc-d3d3-48d9-baa9-738350801c55

My parts arrived during the week, but it wasn't until the weekend that I actually
had time to start putting things together.

However, before I could build the new machine with the new parts, I had to get Cartman out
of the 5U case, which meant moving him into the 4U case I had. The 4U case was populated
with a rather old Linux machine that I hadn't powered up in a couple of years. So
all of that came out, leaving a clean 4U case, ready for loading:

Once the case was clear, out came the rack again and Cartman shut down and pulled.
I put Cartman on a separate table from my regular service table and gradually disassembled
it and moved the parts into the 4U case.

Here you can see Cartman's motherboard loaded into the 4U case. Its an old Tyan board
with dual P3 processors and 512MB of RAM. A great board in its day, its terribly dated
now.

Notice also a pair of PCI-X slots, both of which are normally occupied - one with
the Adaptec
29160 SCSI controller, the other with the Adaptec
2820SA RAID controller. The RAID controller just runs the big SATA storage array,
the boot mirror runs off the SCSI controller, as well as the DVD drive and the external
tape drive.

The drive array sits in the big gap on the left side of the case (right side if you're
looking in the front), and the pair of boot drives live in the little gap full of
wires between the drive array and the DVD.

Mounting the motherboard is always the trickiest bit of the build, once that's done,
the rest goes quickly. The only difference between Cartman in the 5U case and Cartman
in the 4U case is a four drive RAID array instead of six drives.

This
is what Cartman looks like fully loaded into the 4U case and booting up. You can see
the SCSI ribbon cable running to the pair of drives in the center front of the case
and the four blue SATA cables running from the RAID controller to the array chassis.
Even the DVD is SCSI, although where
the drives are SCSI-160
LVD, the DVD is SCSI-40.

The other two cards in Cartman are the video and gigabit ethernet. This is an old
machine, very little is onboard. But you can appreciate why I have to replace Cartman.
All those things we're used to having right on the motherboard have to be added card-by-card.

Cartman was none-the-worse-for-wear in the move, still working the same way, with
just the RAID array being down.

In
fact, as you can see from this shot at the front of the case, I did not install the
terabyte drives into the chassis, since I currently have backup data on three of the
drives, and I need six for the new array in the 5U case.

When I'm finally able to clean off those backup drives (when I believe everything
is stable), I'll build a RAID 1+0 array using the old Adaptec controller. That will
stay under the 2TB limit with the 1TB drives and give me the reliability and performance
I want.

Ultimately, Cartman will be retired, but really only the motherboard. All the drives
are fresh, what's needed is a new multi-core, multi-processor motherboard with a ton
of RAM.

I'm thinking that since the 5U machine has an AMD motherboard, I'll put an Intel board
in this machine. Probably something from ASUS, we'll see.

Cartman went
back into the server closet without incident... tomorrow the 5U case gets a new motherboard,
and a new machine will be born: Butters!

Storage Upgrade Stage 2 - Moving Cartmanhttp://www.campbellassociates.ca/blog/PermaLink,guid,8bf006bc-d3d3-48d9-baa9-738350801c55.aspxhttp://www.campbellassociates.ca/blog/PermaLink,guid,8bf006bc-d3d3-48d9-baa9-738350801c55.aspx
Sun, 14 Oct 2007 01:03:23 GMT<p>
My parts arrived during the week, but it wasn't until the weekend that I actually
had time to start putting things together.
</p>
<p>
However, before I could build the new machine with the new parts, I had to get <a href="http://en.wikipedia.org/wiki/Eric_Cartman">Cartman</a> out
of the 5U case, which meant moving him into the 4U case I had. The 4U case was populated
with a rather old Linux machine that I hadn't powered up in a couple of years. So
all of that came out, leaving a clean 4U case, ready for loading:
</p>
<p>
<a href="http://www.campbellassociates.ca/blog/content/binary/WindowsLiveWriter/FinishingtheStorageUpgrade_2B08/IMG_7860_small.jpg" atomicselection="true"><img style="border-top-width: 0px; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" height="350" alt="The 4U Case Emptied" src="http://www.campbellassociates.ca/blog/content/binary/WindowsLiveWriter/FinishingtheStorageUpgrade_2B08/IMG_7860_small_thumb.jpg" width="526" border="0"></a>
</p>
<p>
Once the case was clear, out came the rack again and Cartman shut down and pulled.
I put Cartman on a separate table from my regular service table and gradually disassembled
it and moved the parts into the 4U case.
</p>
<p>
<a href="http://www.campbellassociates.ca/blog/content/binary/WindowsLiveWriter/FinishingtheStorageUpgrade_2B08/IMG_7863_small.jpg" atomicselection="true"><img style="border-top-width: 0px; border-left-width: 0px; border-bottom-width: 0px; margin: 0px 10px 0px 0px; border-right-width: 0px" height="350" alt="Cartman motherboard in 4U case" src="http://www.campbellassociates.ca/blog/content/binary/WindowsLiveWriter/FinishingtheStorageUpgrade_2B08/IMG_7863_small_thumb.jpg" width="526" align="left" border="0"></a>
</p>
<p>
Here you can see Cartman's motherboard loaded into the 4U case. Its an old Tyan board
with dual P3 processors and 512MB of RAM. A great board in its day, its terribly dated
now.
</p>
<p>
Notice also a pair of PCI-X slots, both of which are normally occupied - one with
the <a href="http://www.adaptec.com/en-US/products/scsi_tech/entry/ASC-29160N/">Adaptec
29160 SCSI controller</a>, the other with the <a href="http://www.adaptec.com/en-US/products/sata_tech/value/AAR-2820SA">Adaptec
2820SA RAID controller</a>. The RAID controller just runs the big SATA storage array,
the boot mirror runs off the SCSI controller, as well as the DVD drive and the external
tape drive.
</p>
<p>
The drive array sits in the big gap on the left side of the case (right side if you're
looking in the front), and the pair of boot drives live in the little gap full of
wires between the drive array and the DVD.
</p>
<p>
Mounting the motherboard is always the trickiest bit of the build, once that's done,
the rest goes quickly. The only difference between Cartman in the 5U case and Cartman
in the 4U case is a four drive RAID array instead of six drives.
</p>
<p>
<a href="http://www.campbellassociates.ca/blog/content/binary/WindowsLiveWriter/FinishingtheStorageUpgrade_2B08/IMG_7864_small.jpg" atomicselection="true"><img style="border-top-width: 0px; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" height="350" alt="Cartman up and running in the 4U case" src="http://www.campbellassociates.ca/blog/content/binary/WindowsLiveWriter/FinishingtheStorageUpgrade_2B08/IMG_7864_small_thumb.jpg" width="526" align="left" border="0"></a>This
is what Cartman looks like fully loaded into the 4U case and booting up. You can see
the SCSI ribbon cable running to the pair of drives in the center front of the case
and the four blue SATA cables running from the RAID controller to the array chassis.
Even the DVD is <a href="http://en.wikipedia.org/wiki/SCSI">SCSI</a>, although where
the drives are <a href="http://en.wikipedia.org/wiki/Single_Connector_Attachment">SCSI-160
LVD</a>, the DVD is SCSI-40.
</p>
<p>
The other two cards in Cartman are the video and gigabit ethernet. This is an old
machine, very little is onboard. But you can appreciate why I have to replace Cartman.
All those things we're used to having right on the motherboard have to be added card-by-card.
</p>
<p>
Cartman was none-the-worse-for-wear in the move, still working the same way, with
just the RAID array being down.
</p>
<p>
<a href="http://www.campbellassociates.ca/blog/content/binary/WindowsLiveWriter/FinishingtheStorageUpgrade_2B08/IMG_7867_small.jpg" atomicselection="true"><img style="border-top-width: 0px; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" height="350" alt="The new drive array" src="http://www.campbellassociates.ca/blog/content/binary/WindowsLiveWriter/FinishingtheStorageUpgrade_2B08/IMG_7867_small_thumb.jpg" width="526" align="left" border="0"></a> &nbsp;In
fact, as you can see from this shot at the front of the case, I did not install the
terabyte drives into the chassis, since I currently have backup data on three of the
drives, and I need six for the new array in the 5U case.
</p>
<p>
When I'm finally able to clean off those backup drives (when I believe everything
is stable), I'll build a RAID 1+0 array using the old Adaptec controller. That will
stay under the 2TB limit with the 1TB drives and give me the reliability and performance
I want.
</p>
<p>
Ultimately, Cartman will be retired, but really only the motherboard. All the drives
are fresh, what's needed is a new multi-core, multi-processor motherboard with a ton
of RAM.
</p>
<p>
I'm thinking that since the 5U machine has an AMD motherboard, I'll put an Intel board
in this machine. Probably something from ASUS, we'll see.
</p>
<p>
<a href="http://animatedtv.about.com/library/weekly/aa040502a.htm">Cartman</a> went
back into the server closet without incident... tomorrow the 5U case gets a new motherboard,
and a new machine will be born: <a href="http://en.wikipedia.org/wiki/Butters_Stotch">Butters!</a>
</p>
<img width="0" height="0" src="http://www.campbellassociates.ca/blog/aggbug.ashx?id=8bf006bc-d3d3-48d9-baa9-738350801c55" />http://www.campbellassociates.ca/blog/CommentView,guid,8bf006bc-d3d3-48d9-baa9-738350801c55.aspxRackmountingToyshttp://www.campbellassociates.ca/blog/Trackback.aspx?guid=54af6eee-5c51-481a-961b-c92ca8b841c2http://www.campbellassociates.ca/blog/pingback.aspxhttp://www.campbellassociates.ca/blog/PermaLink,guid,54af6eee-5c51-481a-961b-c92ca8b841c2.aspxhttp://www.campbellassociates.ca/blog/CommentView,guid,54af6eee-5c51-481a-961b-c92ca8b841c2.aspxhttp://www.campbellassociates.ca/blog/SyndicationService.asmx/GetEntryCommentsRss?guid=54af6eee-5c51-481a-961b-c92ca8b841c2

So if you didn't get the hint,
I'm upgrading the capacity of my servers in my server closet.

Last time I upgraded capacity it was in Cartman,
migrating from a 400GB six drive RAID array to a 2TB six drive RAID array. With the
new 1TB hard drives, I was ready to move to a 2TB six drive RAID array.

So stage 1, actually started back on Friday, was to back everything up... rather than
taking the chance of pulling Cartman and pulling into him directly, I hooked up the
1TB drives, one at a time, to my Phillip, one of my workstations, and copied everything
off. It took until Sunday to finish the copy across three drives.

Here you see one of the drives getting loaded across the network the slow, but
low risk way. Even with gigabit ethernet, transferring data from the old array on
Cartman through the network, into Phillip and then out via SATA takes a long time.

By the way - they may call them 1TB drives, but they format to 933GB. That whole 1000
bytes vs. 1024 bytes thing is getting out of hand. It was fine when we were dealing
with smaller drive, but when you're talking 933GB vs. 1000GB, that's 7% of the capacity
of the drive missing. At some point you have to call foul - this is not a 1TB drive.

So I had three drives filled with the contents of the old array, that left me with
seven drives empty to build the new array... although I only needed six.

Once the backup was finished, it was time to pull Cartman, which meant opening up
the server closet and pulling the rack.

Here's the server rack pulled, Cartman is near the bottom, just above the 2000VA UPS.
The long grey 1U box is an Exabyte 1x10 SCSI tape backup unit. You can also see
the power supply of my temporary Exchange rig that has been running some two years
as just a power supply, motherboard and hard drive sitting on a towel. I'm tempting
fate, I know.

You can see how the rack pulls out on the rails, using folding arms in behind with
cables running across the arms.

Beside the server rack, the second shot is the network rack that has the dual internet
connections, all the patch bay wiring for network, telephone and cable. The 1U console
is pulled out to shut down Cartman, its wired back to the server rack where the
KVM switch is.

Here's a look into Cartman for the first time in a couple of years:

Looks about the same as last time.

That's the end of the photos, because things went downhill from here and I stopped
thinking camera and started thinking much meaner thoughts.

I carefully extracted the six 400GB drives that have been the 2TB array for the past
couple of years. I figured I could always go back to the original drives. I replaced
those drives with six blank 1TB drives. Fired up Cartman and built a new array.

The Adaptec 2810SA controller recognized the drives fine, but wouldn't create an array
bigger than 2.1TB. It appears to be a hard limit of the controller. I upgraded firmware
on the controller, to no avail. I tried configuring it in Windows 2003 Server and
directly in the firmware, hit the same limit either way.

So much for that - now I have to make a choice. I could build two three drive arrays
of 2TB each, or replace the controller. I wasn't going to sacrifice an extra drive
for this, I needed a new controller.

So now that I admitted I needed new hardware, it was time to revisit my thoughts of
hardware migration in general.

The original versions of these servers go all the way back to 2000, with upgrades
on the way. One of the issues I've run into again and again is that migrating to new
servers is hard, so hard that old servers are tough to retire, they just go on and
on until they fail and you're forced to give them up. Cartman, after all, is a dual
P3 machine, still going on. I've upgraded the OS, replaced the CPU fans, swapped the
drives a couple of time... but its still an old machine.

My new vision of the rack is to go to completely virtualized servers. I want to build
a pair of high performance multiple processor servers with lots of RAM and 64 bit
operating systems running multiple virtual machines. I need a pair so that I can fail
between them - they will back each other up and each should be capable of running
the entire server farm itself.

Cartman, obviously, is not qualified for this job. So Cartman will have to go away
eventually.

Through a series of unexpected events that I shall not go into in detail, I ended
up in possession of a Tyan
S2927 motherboard with a pair of AMD dual core processors and a bunch of RAM.
This was a motherboard able to take on my virtualization mission - it just needed
to find a place to live.

The need for a new RAID controller capable of handling arrays bigger than two terabytes
gave me the excuse to make the big move - migrate Cartman out of the 5U case with
six drive bays into the 4U case with four drive bays, and move the new Tyan motherboard
into the case with a controller able to get me my 5TB array.

However, that meant I had to wait for more parts to arrive, which meant waiting a
few days. I'm ordering in more RAM for the S2927 board (might as well fill it) and
an Adaptec
3805, which will go into a PCI-e slot and handle the big array.

I ended my day by putting Cartman back in the rack and back online again, although
without the drive array. We could live without the big storage for a few days.

Upgrading Storage Capacity on Cartmanhttp://www.campbellassociates.ca/blog/PermaLink,guid,54af6eee-5c51-481a-961b-c92ca8b841c2.aspxhttp://www.campbellassociates.ca/blog/PermaLink,guid,54af6eee-5c51-481a-961b-c92ca8b841c2.aspx
Mon, 08 Oct 2007 00:09:23 GMT<p>
So if you didn't get the <a href="http://www.campbellassociates.ca/blog/PermaLink,guid,f1fbc027-d7ee-4e35-8d5a-7adb4a64a289.aspx">hint</a>,
I'm upgrading the capacity of my servers in my server closet.
</p>
<p>
Last time I upgraded capacity it was in <a href="http://www.campbellassociates.ca/blog/PermaLink,guid,32cc0c54-869b-415b-b2a0-531cd40d3eb4.aspx">Cartman</a>,
migrating from a 400GB six drive RAID array to a 2TB six drive RAID array. With the
new 1TB hard drives, I was ready to move to a 2TB six drive RAID array.
</p>
<p>
I bought ten <a href="http://www.seagate.com/ww/v/index.jsp?locale=en-US&amp;name=barracuda-es-2-sata-3-0-gb/s-1-tb-hard-drive&amp;vgnextoid=481e83de34b43110VgnVCM100000f5ee0a0aRCRD&amp;vgnextchannel=b655f141e7f43110VgnVCM100000f5ee0a0aRCRD&amp;reqPage=Model">Seagate
Barracuda ES.2 1TB drives</a>, six for Cartman, and four to go into a different rebuilt
server.
</p>
<p>
So stage 1, actually started back on Friday, was to back everything up... rather than
taking the chance of pulling Cartman and pulling into him directly, I hooked up the
1TB drives, one at a time, to my Phillip, one of my workstations, and copied everything
off. It took until Sunday to finish the copy across three drives.
</p>
<p>
<a href="http://www.campbellassociates.ca/blog/content/binary/WindowsLiveWriter/UpgradingStorageCapacityonCartman_1997/IMG_7840_small.jpg" atomicselection="true"><img style="border-top-width: 0px; border-left-width: 0px; border-bottom-width: 0px; margin: 0px 10px 0px 0px; border-right-width: 0px" height="350" alt="Backing up onto 1TB drives." src="http://www.campbellassociates.ca/blog/content/binary/WindowsLiveWriter/UpgradingStorageCapacityonCartman_1997/IMG_7840_small_thumb.jpg" width="526" align="left" border="0"></a>
</p>
<p>
Here&nbsp;you see one of the drives getting loaded across the network the slow, but
low risk way. Even with gigabit ethernet, transferring data from the old array on
Cartman through the network, into Phillip and then out via SATA takes a long time.
</p>
<p>
By the way - they may call them 1TB drives, but they format to 933GB. That whole 1000
bytes vs. 1024 bytes thing is getting out of hand. It was fine when we were dealing
with smaller drive, but when you're talking 933GB vs. 1000GB, that's 7% of the capacity
of the drive missing. At some point you have to call foul - this is not a 1TB drive.
</p>
<p>
So I had three drives filled with the contents of the old array, that left me with
seven drives empty to build the new array... although I only needed six.
</p>
<p>
Once the backup was finished, it was time to pull Cartman, which meant opening up
the server closet and pulling the rack.
</p>
<p>
<a href="http://www.campbellassociates.ca/blog/content/binary/WindowsLiveWriter/UpgradingStorageCapacityonCartman_1997/IMG_7848_small.jpg" target="_blank" atomicselection="true"><img style="border-top-width: 0px; border-left-width: 0px; border-bottom-width: 0px; margin: 0px 5px 20px 0px; border-right-width: 0px" height="526" alt="Server rack pulled out" src="http://www.campbellassociates.ca/blog/content/binary/WindowsLiveWriter/UpgradingStorageCapacityonCartman_1997/IMG_7848_small_thumb.jpg" width="350" align="left" border="0"></a>
</p>
<p>
<a href="http://www.campbellassociates.ca/blog/content/binary/WindowsLiveWriter/UpgradingStorageCapacityonCartman_1997/IMG_7850_small.jpg" atomicselection="true"><img style="border-top-width: 0px; border-left-width: 0px; border-bottom-width: 0px; margin: 0px 10px 20px 0px; border-right-width: 0px" height="526" alt="Other half of the rack closet" src="http://www.campbellassociates.ca/blog/content/binary/WindowsLiveWriter/UpgradingStorageCapacityonCartman_1997/IMG_7850_small_thumb.jpg" width="350" align="left" border="0"></a>
</p>
<p>
Here's the server rack pulled, Cartman is near the bottom, just above the 2000VA UPS.
The&nbsp;long grey 1U box is an Exabyte 1x10 SCSI tape backup unit. You can also see
the power supply of my temporary Exchange rig that has been running some two years
as just a power supply, motherboard and hard drive sitting on a towel. I'm tempting
fate, I know.
</p>
<p>
You can see how the rack pulls out on the rails, using folding arms in behind with
cables running across the arms.
</p>
<p>
Beside the server rack, the second shot is the network rack that has the dual internet
connections, all the patch bay wiring for network, telephone and cable. The 1U console
is pulled out to shut down Cartman, its wired back to the server rack where&nbsp;the
KVM switch is.&nbsp;
</p>
<p>
&nbsp;
</p>
<p>
Here's a look into Cartman for the first time in a couple of years:
</p>
<p>
<a href="http://www.campbellassociates.ca/blog/content/binary/WindowsLiveWriter/UpgradingStorageCapacityonCartman_1997/IMG_7852_small.jpg" atomicselection="true"><img style="border-top-width: 0px; border-left-width: 0px; border-bottom-width: 0px; border-right-width: 0px" height="350" alt="A naked Cartman!" src="http://www.campbellassociates.ca/blog/content/binary/WindowsLiveWriter/UpgradingStorageCapacityonCartman_1997/IMG_7852_small_thumb.jpg" width="526" align="left" border="0"></a>
</p>
<p>
Looks about the same as last time.
</p>
<p>
That's the end of the photos, because things went downhill from here and I stopped
thinking camera and started thinking much meaner thoughts.
</p>
<p>
I carefully extracted the six 400GB drives that have been the 2TB array for the past
couple of years. I figured I could always go back to the original drives. I replaced
those drives with six blank 1TB drives. Fired up Cartman and built a new array.
</p>
<p>
The Adaptec 2810SA controller recognized the drives fine, but wouldn't create an array
bigger than 2.1TB. It appears to be a hard limit of the controller. I upgraded firmware
on the controller, to no avail. I tried configuring it in Windows 2003 Server and
directly in the firmware, hit the same limit either way.
</p>
<p>
So much for that - now I have to make a choice. I could build two three drive arrays
of 2TB each, or replace the controller. I wasn't going to sacrifice an extra drive
for this, I needed a new controller.
</p>
<p>
So now that I admitted I needed new hardware, it was time to revisit my thoughts of
hardware migration in general.
</p>
<p>
The original versions of these servers go all the way back to 2000, with upgrades
on the way. One of the issues I've run into again and again is that migrating to new
servers is hard, so hard that old servers are tough to retire, they just go on and
on until they fail and you're forced to give them up. Cartman, after all, is a dual
P3 machine, still going on. I've upgraded the OS, replaced the CPU fans, swapped the
drives a couple of time... but its still an old machine.
</p>
<p>
My new vision of the rack is to go to completely virtualized servers. I want to build
a pair of high performance multiple processor servers with lots of RAM and 64 bit
operating systems running multiple virtual machines. I need a pair so that I can fail
between them - they will back each other up and each should be capable of running
the entire server farm itself.
</p>
<p>
Cartman, obviously, is not qualified for this job. So Cartman will have to go away
eventually.
</p>
<p>
Through a series of unexpected events that I shall not go into in detail, I ended
up in possession of a <a href="http://www.tyan.com/product_board_detail.aspx?pid=175">Tyan
S2927 motherboard</a> with a pair of AMD dual core processors and a bunch of RAM.
This was a motherboard able to take on my virtualization mission - it just needed
to find a place to live.
</p>
<p>
The need for a new RAID controller capable of handling arrays bigger than two terabytes
gave me the excuse to make the big move - migrate Cartman out of the 5U case with
six drive bays into the 4U case with four drive bays, and move the new Tyan motherboard
into the case with a controller able to get me my 5TB array.
</p>
<p>
However, that meant I had to wait for more parts to arrive, which meant waiting a
few days. I'm ordering in more RAM for the S2927 board (might as well fill it) and
an <a href="http://www.adaptec.com/en-US/products/sata_tech/value/SAS-3805/">Adaptec
3805</a>, which will go into a PCI-e slot and handle the big array.
</p>
<p>
I ended my day by putting Cartman back in the rack and back online again, although
without the drive array. We could live without the big storage for a few days.
</p>
<img width="0" height="0" src="http://www.campbellassociates.ca/blog/aggbug.ashx?id=54af6eee-5c51-481a-961b-c92ca8b841c2" />http://www.campbellassociates.ca/blog/CommentView,guid,54af6eee-5c51-481a-961b-c92ca8b841c2.aspxRackmountingToyshttp://www.campbellassociates.ca/blog/Trackback.aspx?guid=e3a3dff9-d134-43b7-b366-1a4433dfc824http://www.campbellassociates.ca/blog/pingback.aspxhttp://www.campbellassociates.ca/blog/PermaLink,guid,e3a3dff9-d134-43b7-b366-1a4433dfc824.aspxhttp://www.campbellassociates.ca/blog/CommentView,guid,e3a3dff9-d134-43b7-b366-1a4433dfc824.aspxhttp://www.campbellassociates.ca/blog/SyndicationService.asmx/GetEntryCommentsRss?guid=e3a3dff9-d134-43b7-b366-1a4433dfc824

Decided not to work on Sunday for a change.

Instead, I upgraded servers! Ah, such a geek.

My old web server Stan is
very very old... P3 1Ghz with 512MB of RAM. Running Windows
2000, it has been a workhorse of a machine. I put Stan together
in November of 2000.
Hard to believe it has been essentially running unmodified for over six years. But
that also means those hard
drives have over 50,000 hours on them, which makes them ticking time bombs. And
that's what the SMART
reporting is saying too.

Migrating a web server is a pain in the butt. Lots of little configuration details
you have to get right. To do the testing, I copied a backup of Stan's web sites onto
Jimmy. However, since there are multiple sites on the web server, I depend on host
header identification to sort out what site is what, which means I need to use the
correct names of the web sites to access them. So what's a boy to do? I want to leave
the sites up and running on the old server while I mess around with the new one.

I could have faked out a DNS server, but that seemed like a lot of work. Instead I
modified the HOSTS file on my
main workstation so that the web sites on Jimmy were pointed to directly. Funny how
old technology serves the purpose so well.

Since HOSTS takes priority over any DNS lookup, I was able to point sites (like www.campbellassociates.ca)
to the IP address of Jimmy directly. Then I could tweak and test to my heart's content.

One whammy I ran into was with FrontPage
Server Extensions. For the most part my web server runs the little web sites of
friends and family, and they all use FrontPage, whether
Microsoft wants them to or not. While it set up the extensions easily enough,
I couldn't administer the sites to set up access for the authoring accounts - no matter
what account information I entered, it failed.

Turned out it wasn't me, it was a feature of Windows
Server 2003 Service Pack 1. The service pack added a loopback check, making sure
that the local computer name always matches the host header. And since I'm using multiple
host headers, that's just not going to work. The fix is in Knowledge
Base Article 896861. You have two choices: turn off loopback checking, or enter
all the domain names that are legal for loopback checking.

I turned it off. Call me lazy.

Upgraded dasBlog as well. What I was really
after was Akismet, the comment spam filtering solution.
Unfortunately, the shipping edition of dasBlog doesn't have direct support for it.
But the daily builds have it.
I'm not normally a guy who runs a daily build, but for Akismet, its worth it. Take
that, comment spammers!

Migrating web servers, upgrading dasBlog...http://www.campbellassociates.ca/blog/PermaLink,guid,e3a3dff9-d134-43b7-b366-1a4433dfc824.aspxhttp://www.campbellassociates.ca/blog/PermaLink,guid,e3a3dff9-d134-43b7-b366-1a4433dfc824.aspx
Mon, 21 May 2007 06:05:15 GMT<p>
Decided not to work on Sunday for a change.
</p>
<p>
Instead, I upgraded servers! Ah, such a geek.
</p>
<p>
My old web server <a href="http://www.southparkstudios.com/show/display_char.php?id=3">Stan</a> is
very very old... P3 1Ghz with 512MB of RAM. Running <a href="http://www.microsoft.com/windows2000/default.mspx">Windows
2000</a>, it has been a workhorse of a machine. I put <a href="http://www.southparkstudios.com/downloads/display_image.php?img=http://images.southparkstudios.com/media/images/503/ep_503_02.gif&amp;img_name=Stan on stage" temp_href="http://www.southparkstudios.com/downloads/display_image.php?img=http://images.southparkstudios.com/media/images/503/ep_503_02.gif&amp;img_name=Stan on stage">Stan</a> together
in <a href="http://www.campbellassociates.ca/rack/2000_11.htm">November of 2000</a>.
Hard to believe it has been essentially running unmodified for over six years. But
that also means those <a href="http://www.fujitsu.com/global/support/computing/storage/hdd/eol/dhdd/mpf3xxat-catalog.html">hard
drives</a> have over 50,000 hours on them, which makes them ticking time bombs. And
that's what the <a href="http://en.wikipedia.org/wiki/Self-Monitoring,_Analysis,_and_Reporting_Technology">SMART
reporting</a> is saying too.
</p>
<p>
<a href="http://www.southparkstudios.com/downloads/display_image.php?img=http://images.southparkstudios.com/media/images/308/308_stan.gif&amp;img_name=Stan's pissed" temp_href="http://www.southparkstudios.com/downloads/display_image.php?img=http://images.southparkstudios.com/media/images/308/308_stan.gif&amp;img_name=Stan's pissed">Stan</a> is
just too old to upgrade, he needs to be replaced.
</p>
<p>
His&nbsp;replacement is <a href="http://www.southparkstudios.com/show/display_char.php?id=98">Jimmy</a>,
a machine I already had in the rack that was a testbed for betas of <a href="http://www.microsoft.com/sql/default.mspx">SQL
Server 2005</a>. <a href="http://www.southparkstudios.com/downloads/display_image.php?img=http://images.southparkstudios.com/media/images/606/606_image_25.jpg&amp;img_name=Jimmy auditions" temp_href="http://www.southparkstudios.com/downloads/display_image.php?img=http://images.southparkstudios.com/media/images/606/606_image_25.jpg&amp;img_name=Jimmy auditions">Jimmy</a> is
a P4 3Ghz with 2GB of RAM, running <a href="http://www.microsoft.com/windowsserver2003/default.mspx">Windows
Server 2003 R2 SP2</a>. Takes some time to get used to the <a href="http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/7b037954-441d-4037-a111-94df7880c319.mspx?mfr=true">little
differences between IIS5 and IIS6</a>, but its all bareable.
</p>
<p>
Migrating a web server is a pain in the butt. Lots of little configuration details
you have to get right. To do the testing, I copied a backup of Stan's web sites onto
Jimmy. However, since there are multiple sites on the web server, I depend on host
header identification to sort out what site is what, which means I need to use the
correct names of the web sites to access them. So what's a boy to do? I want to leave
the sites up and running on the old server while I mess around with the new one.
</p>
<p>
I could have faked out a DNS server, but that seemed like a lot of work. Instead I
modified the <a href="http://en.wikipedia.org/wiki/Hosts_file">HOSTS</a> file on my
main workstation so that the web sites on Jimmy were pointed to directly. Funny how
old technology serves the purpose so well.
</p>
<p>
Since HOSTS takes priority over any DNS lookup, I was able to point sites (like <a href="http://www.campbellassociates.ca">www.campbellassociates.ca</a>)
to the IP address of Jimmy directly. Then I could tweak and test to my heart's content.
</p>
<p>
One whammy I ran into was with <a href="http://msdn2.microsoft.com/en-us/office/aa905431.aspx">FrontPage
Server Extensions</a>. For the most part my web server runs the little web sites of
friends and family, and they all use FrontPage, <a href="http://office.microsoft.com/en-us/frontpage/default.aspx">whether
Microsoft wants them to or not</a>. While it set up the extensions easily enough,
I couldn't administer the sites to set up access for the authoring accounts - no matter
what account information I entered, it failed.
</p>
<p>
Turned out it wasn't me, it was a feature of <a href="http://www.microsoft.com/technet/downloads/winsrvr/servicepacks/sp1/default.mspx">Windows
Server 2003 Service Pack 1</a>. The service pack added a loopback check, making sure
that the local computer name always matches the host header. And since I'm using multiple
host headers, that's just not going to work. The fix is in <a href="http://support.microsoft.com/default.aspx?scid=kb;en-us;896861">Knowledge
Base Article 896861</a>. You have two choices: turn off loopback checking, or enter
all the domain names that are legal for loopback checking.
</p>
<p>
I turned it off. Call me lazy.
</p>
<p>
Upgraded <a href="http://www.dasblog.info/">dasBlog</a> as well. What I was really
after was <a href="http://akismet.com/">Akismet</a>, the comment spam filtering solution.
Unfortunately, the shipping edition of dasBlog doesn't have direct support for it.
But the <a href="http://www.dasblog.info/DailyBuilds.aspx">daily builds</a> have it.
I'm not normally a guy who runs a daily build, but for Akismet, its worth it. Take
that, comment spammers!
</p>
<p>
&nbsp;
</p>
<img width="0" height="0" src="http://www.campbellassociates.ca/blog/aggbug.ashx?id=e3a3dff9-d134-43b7-b366-1a4433dfc824" />http://www.campbellassociates.ca/blog/CommentView,guid,e3a3dff9-d134-43b7-b366-1a4433dfc824.aspxASP.NETdasBlogRackmountingSpamhttp://www.campbellassociates.ca/blog/Trackback.aspx?guid=32cc0c54-869b-415b-b2a0-531cd40d3eb4http://www.campbellassociates.ca/blog/pingback.aspxhttp://www.campbellassociates.ca/blog/PermaLink,guid,32cc0c54-869b-415b-b2a0-531cd40d3eb4.aspxhttp://www.campbellassociates.ca/blog/CommentView,guid,32cc0c54-869b-415b-b2a0-531cd40d3eb4.aspxhttp://www.campbellassociates.ca/blog/SyndicationService.asmx/GetEntryCommentsRss?guid=32cc0c54-869b-415b-b2a0-531cd40d3eb48

I've been slowly working my way through the server rack, upgrading all of my servers.
Some of the machines are as much as five years old, and all spinning gear (CPU fans,
case fans, hard drives) are essentially ticking time bombs. In addition there is new
hardware to be added to the rack, which means virtually everything in the rack has
to move... the new configuration with eight servers completely fills the 30U rack.

What makes this especially challenging is that they ARE servers... they're constantly
in use. I can take them down for a few minutes, but after a half hour the phone starts
to ring. However, some servers are more sensitive to this than others - and Cartman
is one of the least sensitive, since its largely an internal-only server.

Cartman has a variety of tasks. Primarily he's a file server, but also a domain controller
(one of two), DHCP and DNS server. As a file server, he has a 400GB RAID array...
doesn't sound like much, but I built it in October
of 2001. Its done with a Promise
SX6000 controller and six 80GB hard drives. At the time, it was a monster.
Since its essentially been on since it was first built those drives have over 30,000
hours of spin time... very scary.

Before tearing Cartman apart I used Acronis True
Image to image the boot drives, and I backed the entire 400GB drive array
up on a single external USB 400GB drive. And yes, I used xcopy with verify and double
checked everything before I tore it down.

This is what I saw after hauling Cartman out of the rack and popping the cover. Essentially
identical to what I saw in October 2001 - one crammed case. You can see the six ATA/100
ribbon cables coming out of the Promise controller running to the two three drive
caddies holding the 80GB drives. In the middle are the two 17GB SCSI drives that are
used as boot drives, which, along with the SCSI DVD drive are run from the Adaptec
29160 SCSI controller. Oh, and an Exabyte external tape drive plugs in there too.

Disassembly of this beast starts with the metal bar running across the case that also
supports the two SCSI hard drives (and a fan). Then the entire front drive array holding
the DVD, floppy and two drive caddies was removed. Both the SCSI and RAID controllers
were pulled as well, leaving the case pretty darn bare. With everything out I powered
up the machine just to take a look and noticed that one of the CPU fans was barely
spinning any more. I had planned on replacing them anyway, this was just extra incentive.

However, the motherboard is so busy that the fancy new Socket 370 cooling blocks I
bought wouldn't even fit in the space! But I was able to use the old blocks by removing
the worn out fans with the the fans from the new blocks.

After a thorough cleaning, I installed a gigabit network card and began the rest of
the reassembly. I'm retiring the Promise controller altogether, going to a SATA array
using six Hitachi
Deskstar 7K400 drives. Yep, that's right... from a 400GB array to 400GB drives,
for a total of two terabytes! And to drive this puppy, I'd need a SATA controller,
so I went back to Adaptec for their 2810SA
controller.

It actually supports eight drives, but I only had space for six, you can see the controller
hard and new caddies to hold the drives. SATA cables are much tidier than ATA cables,
so I got a bunch of space back in the case.

Here you can see the Chenbro caddies with three SATA cables a peice. There's one power
plug for all three drives (which is very nice) and it also has a heavy blower fan
pumping directly onto the drives.

The old 17GB Atlas V drives are replaced with shiny new 147GB
Atlas 10Ks. More disk space!

With everything crammed back in the case, it was time to get things set up. Even before
I started the install of Windows 2003 server I wanted to get the array set up. What
was interesting is that every card installed in the machine had a boot BIOS in it
- the SCSI controller, the RAID controller AND the gigabit network card! Getting the
BIOS set up to boot from the right device took some fiddling.

Then I decided to start the array configuration from the BIOS, so I set up a RAID
5 array. Being a dilligent geek, I went to the Adaptec web site to check for latest
drivers, BIOS updates, and so on. Adaptec had updates for both the 2810SA and the
29160, so I updated both BIOSes. What's stunningly annoying is that you HAVE to install
BIOS updates from a floppy. The software is hard coded to read from drive A and nowhere
else. Presumably I could set up a USB drive to do this, but this old SuperMicro motherboard
ain't that smart.

I was glad I'd checked all this in advance, all over the readme files for the firmware
were warnings that doing these upgrades would destroy the existing arrays, and you'd
need to back everything up. Since I had nothing on the drives, I had nothing to fear.

Feeling smug with all my firmware flashed, I headed off into the BIOS set up for the
2810SA to get my spiffy new drive array configured. Apparently I did it wrong because
I selected “Clean” to start the array rather than “Build/Verify.”

But I didn't know this at the time - off it went, ticking away to itself. I thought
it might take a long time to set up a two terabyte array, but it was done in about
15 minutes... well, almost done. It got to 99% and then said “Controller Kernel
Stopped Running!” And then the machine would reboot. That didn't seem good.

Every time I restarted the machine and went back into the 2810SA BIOS, I'd get the
same error and reboot the machine.

In an effort to be positive about my situation, I ignored the failure and moved on
- set up Windows 2003 Server. Once it was up and running, I tried to install the drivers
for the controller card, but it wouldn't recognize it. That can't be good either.
I filed a tech support request with Adaptec, but wouldn't hear back for 48 hours:
by then I would solve it on my own.

I went to bed late, very grumpy. The next morning I woke up thinking maybe the firmware
update was a mistake. So I reverted - got the old firmware, set up new floppies and
attempted to install it. But it kept failing with the same error. Couldn't revert.

Then, a flash of insight, I realized what was happening to the controller - it was
crashing! And right at the point of completing the array. After it rebooted, the controller
would restart, see the array almost finished configuring and attempt to finish it...
crashing the controller again! So, how to stop the array from rebuilding? Pull all
the hard drives out! That'll slow the bugger down.

Sure enough, as soon as I pulled the drives, I was able to revert the firmware. Why
I still reverted the firmware, I'm not sure - I guess I had a course in mind and thinking
wasn't going to divert it. With the firmware reverted, the array had died, so when
I plugged the drives back in, nothing bad happened.

Now afraid of the BIOS configuration stuff, I booted back into Windows, and reverted
the driver as well to match the firmware. If you've never done this, you're a happier
person than me: reverting to an older driver is a bugger. Windows 2003 Server
has a rollback driver option, but it doesn't work if you haven't previously installed
the older driver. So I had to do this the hardware - uninstall the driver and then
carefully locate all the backup copies of the DLLs and kill them by hand. Once I had
it all, installing the old driver worked, AND it came up just fine.

Now I was able to set up the RAID 5 array from Adaptec's client for Windows, which
was a whole bunch clearer about the right ways to do things. And that's when I discovered
that correctly building a two terabyte array takes an entire day.

The next day I discovered that my two terabyte array is actually a 1.8TB array. And
that Windows understands TB, it displays that way in Windows Explorer. Funny, huh?
I wonder if they have PB (as in petabyte, a thousand terabytes) in there as well.

The rest of the set up was uneventful, really... things got loaded back on, DHCP and
DNS configured, and so on. The next level of excitement would come with the most dangerous
update of all... converting an Exchange 2000 server to 2003!

Rebuilding Cartman...http://www.campbellassociates.ca/blog/PermaLink,guid,32cc0c54-869b-415b-b2a0-531cd40d3eb4.aspxhttp://www.campbellassociates.ca/blog/PermaLink,guid,32cc0c54-869b-415b-b2a0-531cd40d3eb4.aspx
Thu, 24 Feb 2005 02:52:24 GMT<p>
I've been slowly working my way through the server rack, upgrading all of my servers.
Some of the machines are as much as five years old, and all spinning gear (CPU fans,
case fans, hard drives) are essentially ticking time bombs. In addition there is new
hardware to be added to the rack, which means virtually everything in the rack has
to move... the new configuration with eight servers completely fills the 30U rack.
</p>
<p>
What makes this especially challenging is that they ARE servers... they're constantly
in use. I can take them down for a few minutes, but after a half hour the phone starts
to ring. However, some servers are more sensitive to this than others - and Cartman
is one of the least sensitive, since its largely an internal-only server.
</p>
<p>
Cartman has a variety of tasks. Primarily he's a file server, but also a domain controller
(one of two), DHCP and DNS server. As a file server, he has a 400GB RAID array...
doesn't sound like much, but I built it in <a href="http://www.campbellassociates.ca/rack/2001_10.htm">October
of 2001</a>. Its done with a <a href="http://www.promise.com/product/product_detail_eng.asp?segment=RAID%205%20HBAs&amp;product_id=86#">Promise
SX6000&nbsp;controller</a> and six 80GB hard drives. At the time, it was a monster.
Since its essentially been on since it was first built those drives have over 30,000
hours of spin time... very scary.
</p>
<p>
Before tearing Cartman apart I used <a href="http://www.acronis.com">Acronis True
Image</a>&nbsp;to image the boot drives, and I backed the entire 400GB drive array
up on a single external USB 400GB drive. And yes, I used xcopy with verify and double
checked everything before I tore it down.
</p>
<p>
<img src="http://www.campbellassociates.ca/blog/content/binary/P2050020%20(Small).JPG" border=0>
</p>
<p>
This is what I saw after hauling Cartman out of the rack and popping the cover. Essentially
identical to what I saw in October 2001 - one crammed case. You can see the six ATA/100
ribbon cables coming out of the Promise controller running to the two three drive
caddies holding the 80GB drives. In the middle are the two 17GB SCSI drives that are
used as boot drives, which, along with the SCSI DVD drive are run from the <a href="http://www.adaptec.com/worldwide/product/proddetail.html?sess=no&amp;language=English+US&amp;prodkey=ASC-29160&amp;cat=%2fTechnology%2fSCSI%2fSCSI+for+Servers">Adaptec
29160 SCSI controller</a>. Oh, and an Exabyte external tape drive plugs in there too.
</p>
<p>
Disassembly of this beast starts with the metal bar running across the case that also
supports the two SCSI hard drives (and a fan). Then the entire front drive array holding
the DVD, floppy and two drive caddies was removed. Both the SCSI and RAID controllers
were pulled as well, leaving the case pretty darn bare. With everything out I powered
up the machine just to take a look and noticed that one of the CPU fans was barely
spinning any more. I had planned on replacing them anyway, this was just extra incentive.
</p>
<p>
However, the motherboard is so busy that the fancy new Socket 370 cooling blocks I
bought wouldn't even fit in the space! But I was able to use the old blocks by removing
the worn out fans with the&nbsp;the fans from the new blocks.
</p>
<p>
After a thorough cleaning, I installed a gigabit network card and began the rest of
the reassembly. I'm retiring the Promise controller altogether, going to a SATA array
using six <a href="http://www.hitachigst.com/portal/site/hgst/?epi_menuItemID=0cb2780b8274b00ae4c0c516bac4f0a0&amp;epi_menuID=8d237906f078b6fd25ad4e8060e4f0a0&amp;epi_baseMenuID=22f0deefa8f3967dafa0466460e4f0a0">Hitachi
Deskstar 7K400</a> drives. Yep, that's right... from a 400GB array to 400GB drives,
for a total of two terabytes! And to drive this puppy, I'd need a SATA controller,
so I went back to Adaptec for their <a href="http://www.adaptec.com/worldwide/product/proddetail.html?sess=no&amp;language=English+US&amp;prodkey=AAR-2810SA&amp;cat=%2fTechnology%2fSerial+ATA%2fRAID+Controllers">2810SA
controller</a>.
</p>
<p>
<img src="http://www.campbellassociates.ca/blog/content/binary/P2050019%20(Small).JPG" border=0>
</p>
<p>
It actually supports eight drives, but I only had space for six, you can see the controller
hard and new caddies to hold the drives. SATA cables are much tidier than ATA cables,
so I got a bunch of space back in the case.
</p>
<p>
<img src="http://www.campbellassociates.ca/blog/content/binary/P2050021%20(Small).JPG" border=0>
</p>
<p>
Here you can see the Chenbro caddies with three SATA cables a peice. There's one power
plug for all three drives (which is very nice) and it also has a heavy blower fan
pumping directly onto the drives.
</p>
<p>
The old 17GB Atlas V drives are replaced with shiny new <a href="http://www.maxtor.com/portal/site/Maxtor/menuitem.6adb6b8313633595062e6be791346068/?channelpath=/en_us/Products/SCSI%20Hard%20Drives/Atlas%2010K%20Family">147GB
Atlas 10Ks</a>. More disk space!
</p>
<p>
With everything crammed back in the case, it was time to get things set up. Even before
I started the install of Windows 2003 server I wanted to get the array set up. What
was interesting is that every card installed in the machine had a boot BIOS in it
- the SCSI controller, the RAID controller AND the gigabit network card! Getting the
BIOS set up to boot from the right device took some fiddling.
</p>
<p>
Then I decided to start the array configuration from the BIOS, so I set up a RAID
5 array. Being a dilligent geek, I went to the Adaptec web site to check for latest
drivers, BIOS updates, and so on. Adaptec had updates for both the 2810SA and the
29160, so I updated both BIOSes. What's stunningly annoying is that you HAVE to install
BIOS updates from a floppy. The software is hard coded to read from drive A and nowhere
else. Presumably I could set up a USB drive to do this, but this old SuperMicro motherboard
ain't that smart.
</p>
<p>
I was glad I'd checked all this in advance, all over the readme files for the firmware
were warnings that doing these upgrades would destroy the existing arrays, and you'd
need to back everything up. Since I had nothing on the drives, I had nothing to fear.
</p>
<p>
Feeling smug with all my firmware flashed, I headed off into the BIOS set up for the
2810SA to get my spiffy new drive array configured. Apparently I did it wrong because
I selected &#8220;Clean&#8221; to start the array rather than &#8220;Build/Verify.&#8221;
</p>
<p>
But I didn't know this at the time - off it went, ticking away to itself. I thought
it might take a long time to set up a two terabyte array, but it was done in about
15 minutes... well, almost done. It got to 99% and then said &#8220;Controller Kernel
Stopped Running!&#8221; And then the machine would reboot. That didn't seem good.
</p>
<p>
Every time I restarted the machine and went back into the 2810SA BIOS, I'd get the
same error and reboot the machine.
</p>
<p>
In an effort to be positive about my situation, I ignored the failure and moved on
- set up Windows 2003 Server. Once it was up and running, I tried to install the drivers
for the controller card, but it wouldn't recognize it. That can't be good either.
I filed a tech support request with Adaptec, but wouldn't hear back for 48 hours:
by then I would solve it on my own.
</p>
<p>
I went to bed late, very grumpy. The next morning I woke up thinking maybe the firmware
update was a mistake. So I reverted - got the old firmware, set up new floppies and
attempted to install it. But it kept failing with the same error. Couldn't revert.
</p>
<p>
Then, a flash of insight, I realized what was happening to the controller - it was
crashing! And right at the point of completing the array. After it rebooted, the controller
would restart, see the array almost finished configuring and attempt to finish it...
crashing the controller again! So, how to stop the array from rebuilding? Pull all
the hard drives out! That'll slow the bugger down.
</p>
<p>
Sure enough, as soon as I pulled the drives, I was able to revert the firmware. Why
I still reverted the firmware, I'm not sure - I guess I had a course in mind and thinking
wasn't going to divert it. With the firmware reverted, the array had died, so when
I plugged the drives back in, nothing bad happened.
</p>
<p>
Now afraid of the BIOS configuration stuff, I booted back into Windows, and reverted
the driver as well to match the firmware. If you've never done this, you're a happier
person than me: reverting to an older driver is a bugger.&nbsp;Windows 2003 Server
has a rollback driver option, but it doesn't work if you haven't previously installed
the older driver. So I had to do this the hardware - uninstall the driver and then
carefully locate all the backup copies of the DLLs and kill them by hand. Once I had
it all, installing the old driver worked, AND it came up just fine.
</p>
<p>
Now I was able to set up the RAID 5 array from Adaptec's client for Windows, which
was a whole bunch clearer about the right ways to do things. And that's when I discovered
that correctly building a two terabyte array takes an entire day.
</p>
<p>
The next day I discovered that my two terabyte array is actually a 1.8TB array. And
that Windows understands TB, it displays that way in Windows Explorer. Funny, huh?
I wonder if they have PB (as in petabyte, a thousand terabytes) in there as well.
</p>
<p>
The rest of the set up was uneventful, really... things got loaded back on, DHCP and
DNS configured, and so on. The next level of excitement would come with the most dangerous
update of all... converting an Exchange 2000 server to 2003!
</p>
<img width="0" height="0" src="http://www.campbellassociates.ca/blog/aggbug.ashx?id=32cc0c54-869b-415b-b2a0-531cd40d3eb4" />http://www.campbellassociates.ca/blog/CommentView,guid,32cc0c54-869b-415b-b2a0-531cd40d3eb4.aspxRackmountingToyshttp://www.campbellassociates.ca/blog/Trackback.aspx?guid=db568b13-57ee-419f-ad28-e19c73f28875http://www.campbellassociates.ca/blog/pingback.aspxhttp://www.campbellassociates.ca/blog/PermaLink,guid,db568b13-57ee-419f-ad28-e19c73f28875.aspxhttp://www.campbellassociates.ca/blog/CommentView,guid,db568b13-57ee-419f-ad28-e19c73f28875.aspxhttp://www.campbellassociates.ca/blog/SyndicationService.asmx/GetEntryCommentsRss?guid=db568b13-57ee-419f-ad28-e19c73f288757

Well, I finally broke down and started to rework my racks. I've literally avoided
pulling them for more than a year, just patching things together whichever way I could.
Take a look at the mess they were in before I started:

Several highlights of this mess I call my racks... notice the two bars poking out
the front, those are the rails that the entire rack slide out on. Notice that between
the two racks there's a new server (named “Tweak”) that has been sitting
like that for six months. And notice the freakshow of a wiring mess as I've added
VOIP boxes, a new router, new wireless access point (sitting on top of Tweak), and
so on. Hey, its been more than a year!

The racks themselves are 30U
Middle Atlantic AXS racks. The left hand one is for networking, it has a cable
channel mounted on the left side for all the wiring. On the right is the server rack,
which I had modified to be 30 inches deep instead of the standard 20 inches that Middle
Atlantic makes for these racks. They're intended for stereo equipment, I use them
for the computer gear because this way the server closet is much smaller - you don't
need room to walk around it.

This is the rack pulled out onto the rails and ready for some service work. You can
see the cable channel clearly now.

From the other side you can see the mess of wiring strung between the two racks...
and the mess of wire in the back. Its not as bad as it looks (which is good, it looks
pretty bad). Notice also the “wall-shaker“ style air conditioner that
keeps the whole closet cool.

Besides the tangled mess of wiring, I also needed to add more power plugs, re-arrange
some components, add new gigabit switches and additional wiring between the two racks.

A couple of hours later, the mess of wires is gone from the rack. This shot also shows
the new double-sided power bar I added at the back to give myself more outlets, and
the Oregon Scientific wireless temperature sensor (reading 71.6F) that lets me know
the temperature inside the closet. Normally its about 68F in there. There are alarms
if it climbs above 75F. Also, this gives you a pretty good look at the folding arms
that hold the rack from sliding off the end of the rails, and provide a channel to
route the wires on and off the rack.

Here's the beauty shot of the network rack reconfigured and back in the closet. Here's
an inventory (from top-to-bottom):

That one bright green Ethernet cable you see in the shot is the patch cable for Tweak,
the server still sitting on its side between the racks. I ran a new patch for it through
the rack properly.

Next up, the server rack! And believe me, the network rack was the easy part of this
whole process.

Rack Attack!http://www.campbellassociates.ca/blog/PermaLink,guid,db568b13-57ee-419f-ad28-e19c73f28875.aspxhttp://www.campbellassociates.ca/blog/PermaLink,guid,db568b13-57ee-419f-ad28-e19c73f28875.aspx
Wed, 16 Feb 2005 03:41:09 GMT<p>
Well, I finally broke down and started to rework my racks. I've literally avoided
pulling them for more than a year, just patching things together whichever way I could.
Take a look at the mess they were in before I started:
</p>
<p align=center>
<img src="http://www.campbellassociates.ca/blog/content/binary/P1290019%20(Custom).JPG" border=0>
</p>
<p>
Several highlights of this mess I call my racks... notice the two bars poking out
the front, those are the rails that the entire rack slide out on. Notice that between
the two racks there's a new server (named &#8220;Tweak&#8221;) that has been sitting
like that for six months. And notice the freakshow of a wiring mess as I've added
VOIP boxes, a new router, new wireless access point (sitting on top of Tweak), and
so on. Hey, its been more than a year!
</p>
<p>
The racks themselves are <a href="http://www.middleatlantic.com/enclosure/roll/axsm.htm">30U
Middle Atlantic AXS</a> racks. The left hand one is for networking, it has a cable
channel mounted on the left side for all the wiring. On the right is the server rack,
which I had modified to be 30 inches deep instead of the standard 20 inches that Middle
Atlantic makes for these racks. They're intended for stereo equipment, I use them
for the computer gear because this way the server closet is much smaller - you don't
need room to walk around it.
</p>
<p align=center>
<img src="http://www.campbellassociates.ca/blog/content/binary/P1290020%20(Custom).JPG" border=0>
</p>
<p>
This is the rack pulled out onto the rails and ready for some service work. You can
see the cable channel clearly now.
</p>
<p align=center>
<img src="http://www.campbellassociates.ca/blog/content/binary/P1290021%20(Custom).JPG" border=0>
</p>
<p>
From the other side you can see the mess of wiring strung between the two racks...
and the mess of wire in the back. Its not as bad as it looks (which is good, it looks
pretty bad). Notice also the &#8220;wall-shaker&#8220; style air conditioner that
keeps the whole closet cool.
</p>
<p>
Besides the tangled mess of wiring, I also needed to add more power plugs, re-arrange
some components, add new gigabit switches and additional wiring between the two racks.
</p>
<p align=center>
<img src="http://www.campbellassociates.ca/blog/content/binary/P1300023%20(Small).JPG" border=0>
</p>
<p>
A couple of hours later, the mess of wires is gone from the rack. This shot also shows
the new double-sided power bar I added at the back to give myself more outlets, and
the Oregon Scientific wireless temperature sensor (reading 71.6F) that lets me know
the temperature inside the closet. Normally its about 68F in there. There are alarms
if it climbs above 75F. Also, this gives you a pretty good look at the folding arms
that hold the rack from sliding off the end of the rails, and provide a channel to
route the wires on and off the rack.
</p>
<p align=center>
<img src="http://www.campbellassociates.ca/blog/content/binary/P1300024%20(Custom).JPG" border=0>
</p>
<p>
Here's the beauty shot of the network rack reconfigured and back in the closet. Here's
an inventory (from top-to-bottom):
</p>
<ul>
<li>
Gear shelf contains
<ul>
<li>
D-Link 300i DSL modem
<li>
<a href="http://www.smarthome.com/7702st1.html">Channel Vision Stereo Modulator</a>&nbsp;(puts
my front/back door cameras onto channel 65/68)
<li>
<a href="http://www.dlink.com/products/?pid=14">D-Link DWL-7000AP</a> 802.11a/b/g
wireless access point
<li>
Two <a href="http://broadband.motorola.com/noflash/vt1000.html">Motorola VT1000</a> VOIP
boxes (from <a href="http://www.vonage.ca/">Vonage</a>)
<li>
Hayes Fax Modem
<li>
Cable Modem</li>
</ul>
<li>
<a href="http://www.xincom.com/twr603.html">Xincom 603</a> Dual WAN NAT router
<li>
<a href="http://www.linksys.com/products/product.asp?grid=35&amp;scid=42&amp;prid=569">Linksys
SR2024</a> 24 port Gigabit switch
<li>
2U cable tray
<li>
2U 48 port Ethernet patch panel
<li>
2U cable tray
<li>
<a href="http://www.linksys.com/products/product.asp?grid=35&amp;scid=42&amp;prid=570">Linksys
SR224G</a> 24 port 10/100 switch (with Gigabit uplink)
<li>
1U Keyboard/Mouse/Monitor console
<li>
Cisco 3620 (mounted backwards)
<li>
3U 48 port keystone patch panel (telephone and cable patches)
<li>
The old Nexland dual WAN NAT router
<li>
5U gap (more UPSes will go in here in the future)
<li>
1U power bar
<li>
3U Hewlett-Packard rack-mount oscilloscope (long story)
<li>
<a href="http://www.sizemyups.com/specs/1011/90000519/BR/0/">2U Minuteman 1000VA UPS</a> (cut
off in the photo)</li>
</ul>
<p>
That one bright green Ethernet cable you see in the shot is the patch cable for Tweak,
the server still sitting on its side between the racks. I ran a new patch for it through
the rack properly.
</p>
<p>
Next up, the server rack! And believe me, the network rack was the easy part of this
whole process.
</p>
<p align=center>
<img src="http://www.campbellassociates.ca/blog/content/binary/P1300026%20(Custom).JPG" border=0>
</p>
<img width="0" height="0" src="http://www.campbellassociates.ca/blog/aggbug.ashx?id=db568b13-57ee-419f-ad28-e19c73f28875" />http://www.campbellassociates.ca/blog/CommentView,guid,db568b13-57ee-419f-ad28-e19c73f28875.aspxRackmountingToyshttp://www.campbellassociates.ca/blog/Trackback.aspx?guid=d17f786f-adca-4919-8816-1dbdebabd22dhttp://www.campbellassociates.ca/blog/pingback.aspxhttp://www.campbellassociates.ca/blog/PermaLink,guid,d17f786f-adca-4919-8816-1dbdebabd22d.aspxhttp://www.campbellassociates.ca/blog/CommentView,guid,d17f786f-adca-4919-8816-1dbdebabd22d.aspxhttp://www.campbellassociates.ca/blog/SyndicationService.asmx/GetEntryCommentsRss?guid=d17f786f-adca-4919-8816-1dbdebabd22d1

On June 24th I was a guest on DotNetRocks...
but we didn't talk about .NET, we talked about my favorite subject, TOYS! Actually,
the focus was on water-cooled computers, which is definitely toy-ish, although we
digressed into a number of equally entertaining topics.

There were a variety of questions, so I figured I'd best answer them here. First off,
I put together a little photo-pictorial of one of my
water cooling conversions.

One of the gizmos I used in that photo-pictorial but didn't take a photo of is this
little motherboard power adapter that I plug into my power supply so that I can fire
it up without having to actual turn the machine on. It's very useful for being able
to run the pump without heating anything delicate up.

Its just a female 20 pin ATX plug that connects pins 13 (ground) and 14 (power supply
on) together. So there ya go Geoff, don't say I never did nuthin fer ya.

If you looked at the water cooling page above, you may have noticed I'm using rackmount
cases for my workstations. I have a server closet that's all rackmounted, but I also
had my desk custom built with rackmount bays as well.

This is one of the bays being fitted out while the office was still under construction.
The rack itself is a 12U Middle
Atlantic SRSR Rotating Sliding Rail System rack. This rack actually slides out
of the bay and then rotates once fully extended so you can get at the back of the
case without digging around blind. I have two of these in the office, one for each
main workstation bay. There's enough room on the rack for a UPS, two PCs and other
sundry gear.

Although we didn't talk a whole lot about it on the show, for rackmount junkies, here
are a couple of links to my server rack set ups. The first link is to my old
rack, which ran from September 2000 to December 2002. After that, my new
rack server closet was up and running, which is how it continues to this day.

This shot was taken today... the rack is essential the same as it was December 19,
2002, except that it's a whole bunch messier. Over the summer I'll be rebuilding most
of the systems in here, after all, some of the hard drives are now four years old
an essentially ticking time bombs well past their MTBF (Mean Time Before Failure).

Doing DotNetRocks!http://www.campbellassociates.ca/blog/PermaLink,guid,d17f786f-adca-4919-8816-1dbdebabd22d.aspxhttp://www.campbellassociates.ca/blog/PermaLink,guid,d17f786f-adca-4919-8816-1dbdebabd22d.aspx
Mon, 28 Jun 2004 01:50:16 GMT<p>
On June 24th I was a guest on <a href="http://www.franklins.net/dotnetrocks/">DotNetRocks</a>...
but we didn't talk about .NET, we talked about my favorite subject, TOYS! Actually,
the focus was on water-cooled computers, which is definitely toy-ish, although we
digressed into a number of equally entertaining topics.
</p>
<p>
There were a variety of questions, so I figured I'd best answer them here. First off,
I put together a little <a href="/watercooling/">photo-pictorial</a> of one of my
water cooling conversions.
</p>
<p>
One of the gizmos I used in that photo-pictorial but didn't take a photo of is this
little motherboard power adapter that I plug into my power supply so that I can fire
it up without having to actual turn the machine on. It's very useful for being able
to run the pump without heating anything delicate up.
</p>
<p>
<img src="/blog/content/binary/IMG_6590 (Small).JPG" border=0>
</p>
<p>
Its just a female 20 pin ATX plug that connects pins 13 (ground) and 14 (power supply
on) together. So there ya go Geoff, don't say I never did nuthin fer ya.
</p>
<p>
If you looked at the water cooling page above, you may have noticed I'm using rackmount
cases for my workstations. I have a server closet that's all rackmounted, but I also
had my desk custom built with rackmount bays as well.
</p>
<p>
<img src="/blog/content/binary/IMG_5830 (Custom).JPG" border=0>
</p>
<p>
This is one of the bays being fitted out while the office was still under construction.
The rack itself is a 12U <a href="http://www.middleatlantic.com/enclosure/roll/srsr.htm">Middle
Atlantic SRSR Rotating Sliding Rail System</a> rack. This rack actually slides out
of the bay and then rotates once fully extended so you can get at the back of the
case without digging around blind. I have two of these in the office, one for each
main workstation bay. There's enough room on the rack for a UPS, two PCs and other
sundry gear.
</p>
<p>
Although we didn't talk a whole lot about it on the show, for rackmount junkies, here
are a couple of links to my server rack set ups. The first link is to my <a href="/rack/">old
rack</a>, which ran from September 2000 to December 2002. After that, my <a href="/newrack/">new
rack server</a> closet was up and running, which is how it continues to this day.
</p>
<p>
<img src="/blog/content/binary/IMG_6591 (Small).JPG" border=0>
</p>
<p>
This shot was taken today... the rack is essential the same as it was December 19,
2002, except that it's a whole bunch messier. Over the summer I'll be rebuilding most
of the systems in here, after all, some of the hard drives are now four years old
an essentially ticking time bombs well past their MTBF (Mean Time Before Failure).
</p>
<img width="0" height="0" src="http://www.campbellassociates.ca/blog/aggbug.ashx?id=d17f786f-adca-4919-8816-1dbdebabd22d" />http://www.campbellassociates.ca/blog/CommentView,guid,d17f786f-adca-4919-8816-1dbdebabd22d.aspxRackmountingSpeakingToysWater Cooling