From sprinklers to battlestations: Ars staffers’ crazy home lab experiments

Perhaps you know someone, dear reader, who is what they call a “car person.” This “car person” perhaps has many cars in various states of repair, or perhaps there is one “special” car the “car person” lovingly works on without ever actually driving it—changing out the “engine” or the “pistons,” endlessly replacing the “tie rods,” constantly benchmarking new “oil sumps,” or…OK, I’m not actually a “car person” myself and so what that species does with cars is a mystery, but you get the idea: perhaps you know someone who tinkers on automobiles.

Well, many of us Ars Technica staffers are like that—except, perhaps unsurprisingly, instead of cars we tinker on computers. And usually not just with any one computer—the lure of getting your hands dirty and building something functional and performant and cool transcends any one personality or interest type, be it cars or computers. Much like the car person with the Datsun Fairlady Z-car small block V8 conversion perpetually under work in the garage, we geeks tend to have our own perpetually in-work projects—improving our workstations, building tools, or even crafting our own mad science laboratories in our homes. That those labs are most often used for things like “deploying a hundred virtual machines and then simulating multiple states of degraded network performance between each of them” and not “creating an undead monster” doesn’t lessen the craziness of what we’re doing—after all, much like Doctor Frankenstein, our motivations for tinkering at home are rarely monetary and often instead based on curiosity and a drive to create.

Not everyone on staff has a needlessly complex IT setup, of course—Andrew Cunningham responded to our request for input on this staffsource with an eye-roll so loud that I could actually hear it through email, and I’m pretty sure Jon Brodkin actually thinks “cat 5” refers to the fifth such animal in a group of felines. But the true nutters on staff stood tall and provided us some quality descriptions of their creations.

So, while you Americans relax on your couch in post-turkey bliss or hide in the bathroom so you don’t have to talk about politics with your family—and while the rest of the world goes about its normal Thursday business—we present to you five crazier-than-they-need-to-be Ars Technica staffer home IT setups.

Cyrus Farivar: Singing in the rain

Unlike my esteemed colleagues, I don’t have much in the way of crazy home IT setups. I mean, sure, I have a few basic hard drives attached to my Apple Airport Extremes that serve as largely unused NASes, but the most interesting is my OpenSprinkler.

Enlarge / A Raspberry Pi OpenSprinkler, tucked away on some water pipes, for now.

Cyrus Farivar

It’s been nearly two years since I set up my Raspberry Pi to act as the engine behind our family’s drip system for our modest garden. (I detailed the saga back in February 2015.)

This past summer, when we began a major cull of our garden (and my wife single-handedly tore out a mess over some overgrown plants), I was tasked with mapping our existing in-ground drips, re-laying new drips where necessary, and giving our Pi some love. The Pi was doing just fine, doing its thing. To that end, I ended up re-installing the PiOS on it in the name of spring/summer cleaning.

As we’ve been doing some remodeling in our house in recent months, I’ve had to sort of precarious position it on some overhead pipes, which has meant that the tiny little power cord pops out sometimes—but I’ll mount it back on the wall before the end of the year. I’m still trying to figure out if there’s anything else that’s both fun and useful that I can get my Pi do to in its current configuration.

Eric Bangeman: My office, my life

If I had contributed to this article a decade ago, I would’ve written about how I hardwired my 1929-vintage house with Cat-5e cable and used a gigabit switch for all of my devices (three or four desktop PCs and a console). But in 2008, we moved to a newer house, and advances in wireless networking dampened any enthusiasm I had for running cable through my walls, then patching and repainting over the mess.

Some people have a server rack. I have a server bookshelf. Functioning hardware from left to right, top to bottom: Synology DS416 NAS with 11TB of HDDs installed, Amplifi HD router, HP Color LaserJet Pro 476dw multifunction printer, UPS, Motorola cable modem (owned, not leased), and Ooma VoIP box. The G4 Cube up top used to function as a home server until it crapped out.

I own a Motorola cable modem through which my ISP, Wide Open West, happily pumps 100mbps/10mbps service. Hooked up to the modem is my Ubiquiti Amplifi HD router. That in turn has four Ethernet ports in back to which I have connected my office iMac, my SIP phone, my NAS, and my Ooma VoIP box for home phone service hooked up. Everything else in the house uses Wi-Fi, which propagates through our home thanks to the Amplifi HD’s mesh antennas.

On the wired network:

Synology DS416 NAS. This is my most recent hardware purchase. I’m using it for backing up all of the laptops as well as for syncing files between my wife’s iMac and laptop. The drives are a combination of 3TB Western Digital Red HDDs and a couple of used WD Black drives I had lying around.

Ooma VoIP device. Not everyone in the Bangeman household likes carrying a smartphone around all of the time, and at $5 per month, Ooma offers what we need for residential phone service. Of course, we are down to one or two non-robocalls per week…

Polycom SIP phone. All Ars staffers have a SIP phone with a three-digit extension on their desks.

2015 Retina 5K iMac with a 4GHz Intel Core i7 CPU and 24GB of DDR3 RAM. To avoid unsightly cable runs, the Ethernet cable for the phone and iMac are routed through conduit installed just above the baseboards in my office.

On the wireless network:

Two Apple TVs

One Roku HD

HP Color LaserJet Pro MFP M476dw. This is a multifunction (print/scan/fax) color laser printer from HD. If you need to print stuff regularly (and we do) and don’t like paying thousands of dollars per gallon of printer ink (and we don’t), this is a fine piece of hardware. Ignore the low-toner warnings (seriously, ignore them until you see missing colors on your printouts), and you’ve got a sweet multifunction device that scans, copies, and prints extremely fast.

Six laptops (including two Chromebooks and a Surface Pro)

DirecTV HD DVR

Various and sundry iPads and Kindles

Samsung Smart TV (used for Netflix and Amazon Prime Video)

If I had written this blurb three months ago, I would’ve whined about poor wireless signal in some parts of the house and vacillated over whether I should have done an Ethernet cable run after all. But since I replaced my Apple Time Capsule and Airport Express with the Ubiquiti gear, I am perfectly happy with my setup. Next on the horizon are some IoT devices like cameras and maybe a thermostat. But I don’t want my devices incorporated into a botnet, so I’m going to hold off on diving into the Internet of Things for the time being.

Annalee Newitz: A fount of Largesse and a wellspring of terabytes

In my home there is a very special media server named Largesse. You see, I have a rather large collection of extremely strange VHS, CD, DVD, and Blu-Ray media. If one were feeling generous, one might say that they are disorganized. Which is why my beloved housemate helped me to create a special data palace to hold them all. Here’s how we did it (with mad props to Gregor Menasian, who taught us almost everything we know about extremely large and cheap data storage boxes).

I never promised you that my server closet would be perfectly organized. Hey, there are shelves. Isn’t that enough?

You can see Largesse, the media server, in the lower right. In the upper left you can glimpse our backup server. Please do not comment on dust or clutter or other stupid human concerns.

Now you’re getting down on your knees to see the good stuff. About this time you’re saying something like, “So, Ubuntu and ZFS. Tell me about your ZFS feelings.”

Here it is, baby. Look at that. Fifteen four-terabyte hard drives. Blinky lights. Really loud fans that you can hear in the next room. BUT IT’S ALL WORTH IT.

It’s our backup server! Eighteen two-terabyte drives.

Here’s an important corner of our 4K display, with streamer to the right.

Right now we’re using an Nvidia Shield as our Android player. Also there is my original VHS copy of Star Wars. No CGI added!

This is part of my movie and TV collection. It might be considered large and disorganized by people who haven’t passed through the Singularity.

A closer look at my superlative organization system. It took about a month to digitize everything, though that was just working on and off when I had time.

I just wanted you to understand that each of those shelves is actually TWO LAYERS of DVDs. So when you look at that shelf, you are only scratching the surface. This is why I needed a media server.

First, allow me to introduce you to Largesse. It’s a beautiful machine with a slightly older Intel E3 CPU that supports ECC memory. Currently it has fifteen HGST 3.5 inch 4TB hard drives formatted with ZFS, full of digital backups of all our movies, TV, music, books, home software projects, and other sundry items. It has 32GB of ECC RAM, which I’ll explain in a minute. Largesse runs Ubuntu, and we access it remotely with SSH and use Samba to share its contents across various Android players around the house, which play video using Archos or Kodi. That’s the executive summary.

To run all those disks, we got two very cheap, hackable controller cards: IBM ServerRaid M1015s, which plug into PCI eXpress 2.0 x8 slots on the motherboard. We needed a controller that gave us enough SATA interfaces, and the M1015 was the ticket. Normally when you plug in one of these cards, you have the controller take care of the disks’ RAID configuration, but that wouldn’t have worked with our goal of using ZFS. So we configured the controllers to function in JBOD mode—which literally stands for “just a bunch of disks.” This exposes all of the disks directly to the operating system instead of trying to present them as a RAID volume.

We decided to use ZFS because if you have 15 disks with one file system, you need a lot of flexibility about your storage pool. Also, as beloved housemate puts it, “ZFS is badass.”

With that many drives, cabling is a nightmare. The great thing about M1015s is that they also have two SFF-8087 multi-lane connectors, which can each be expanded to provide four SATA connectors for disks. So you can technically run 16 disks through two M1015s. We have eight on one, and seven on the other. It’s beautiful and makes us happy.

A quick word about our hard drives: We chose the HGST drives because they offered the highest density at a low failure rate. We run them in a Z2 configuration, which means we won’t lose any data unless three drives fail. We have yet to see any drives fail in over two years. Also, we have those 32 gigs of RAM because ZFS likes RAM. Plus, when you get a large amount of memory the likelihood of some kind of bit-flip event increases, so error-correcting memory goes a long way toward helping keep data loss to a minimum—it will resist random corruptions better than other memory.

Obviously we run open source software. We migrated to Ubuntu from NAS4Free a while ago, and the migration was ridiculously easy. Ubuntu has a good ZFS implementation and gives us more flexibility about the software we run. We have a few ways of accessing the data on the server, via our two home networks (both gigabit Ethernet NICs). We run a Plex server that lets us stream stuff easily from tablets and phones. As I mentioned earlier, we do remote administration via SSH. We also use SSH for file transfers after we’ve created a digital backup from a new DVD or Blu-Ray or whatever. We use Samba to share files to Android players. And we play video with either Kodi or Archos—whichever one isn’t being annoying and crashing that day.

The nice thing is that we can also use Chromecast with the Android players if we want to stream something to our TV. Currently we’re using an Nvidia Shield for our 4K display. We also have a system for casting from Largesse through a phone app to a projector in beloved housemate’s bedroom. Explaining that would take another article, though, so let’s not go there.

One of the questions people always ask is how we back up our exceedingly large collection of media. Luckily, Largesse has a predecessor. It’s an 8-disk QNAP box that has two five-disk DAS connected by eSATA. Once its eighteen 2TB disks were enough for us—but now it lives on only as a backup for Largesse.