A story on Slashdot yesterday encouraged IT departments to hire a hacker, in spite of the stigma.

I’ve been that guy, and I suspect I’ll be that guy again. I’ve also had to clean up after that guy, so I may be able to add some perspective.
One trouble is that the word “hacker” means a lot of different things to different people. There’s a certification, Certified Ethical Hacker, that does not mean what this article is talking about. So you need to make sure you know the kind of hacker you’re looking for, and hire the right kind. Hackers aren’t interchangeable, just like chiropractors and dentists and heart surgeons aren’t interchangeable, even though they’re all doctors.

And that’s a serious distinction. I have a colleague who’s a Certified Ethical Hacker. We have him look at security audits, and we’ve had him do some intelligence gathering for us. He’s very good at what he does. But when it comes to building a WordPress plugin that checks the Copyscape API, one of the things the article touts as something a hacker can do….? He’d be lost.

I’m probably closer to what the article means when they say a hacker. I’ve been the guy who roams from project to project, fixing annoying things without saying much, writing scripts to automate repetitive tasks, working without close supervision and not liking close supervision when I had it. I have a fake “Help Wanted” ad from an evil genius in my cubicle, and I want all of my coworkers to think I listen to GWAR. And I’m outspoken.

My current boss actually does take me to meetings, but I rarely speak. Mostly I’m there to tell him who knows what they’re talking about, who’s full of it, and to feed him information.

And since I’ve been job hunting recently, I’ve been speaking with former colleagues, collecting references. I see a pattern, talking to them. People notice when I’m gone. I couldn’t fix everything, but in every case save one, I was fixing a lot more than anybody ever realized. (I worked a job for six months in 2005 where all I did was follow procedures. I showed up for work every day–sober and alert even!–and I did everything that was asked of me, but didn’t excel.)

The key is to look for that hacker mentality and the right skill set. If you need a guy who can write quick and dirty scripts in Perl, ask that in the interview. And if you’re a hiring manager and don’t know the difference between Perl and PHP and Powershell, bring along a technie who does to the interview. Your techie doesn’t necessarily have to speak a lot.

If what you want is someone who can make a copy of your production system, then sit down with it and test its security, keep in mind that may or may not be the same guy or gal. So don’t assume–ask during the interview.

I struggled in 2002-2005 largely because I was working for people who wanted me to automate and script things that weren’t designed to be scripted. In late 2005, I landed in a similar environment, and once I knew the rules and expectations, I thrived there. After I left in 2009, my former coworkers quickly tired of hearing the boss say, “Dave never had any trouble doing that.”

Now, let’s talk caveats.

In the late 1990s, I worked with a guy who certainly lived on the edge between ethical and unethical hacking. He got the job because he was willing to work cheap. His start in IT came when he was in college, and the IT department caught him hacking into their file server and copying all of the software there. The school disciplined him, but the IT department hired him. From there, he jumped to the school where I was working.

He was versatile and fairly talented. His work ethic wasn’t what it could have been, but he was only 21 or 22 at the time. He could have grown into that.

His biggest problem was that he wasn’t honest. One night at around midnight, he called me. The print server wasn’t working, and we were facing a production deadline. It was pretty clear very quickly that he didn’t want the server to start working. I started to walk him through some troubleshooting steps and he wouldn’t cooperate. I’d have him type a command and ask him what the output was, and he’d always say, “nothing.” I knew he was lying because the answer never was “nothing.” If it worked, the server would tell him, and if it didn’t work, the server would give an error message. I ended up telling him to print to another printer and/or direct-connect a printer to one of the computers and print from it in order to meet deadline, and we’d fix the server right the next day.

I came in the next morning, told one of my other colleagues what had happened, and asked him to come with me when I investigated. If he’d been lying, I was going to need a witness. If he was telling the truth, we had a really sick server on our hands and I was going to need someone to help me formulate a get-well plan.

We found the server sitting there with an administrative account still logged in and the command prompt window open from the night before. That was the end of the investigation. He’d been lying. Plain as day, the machine had responded as expected to the commands I had him type in, and there was no reason why the printer shouldn’t have been working.

And as I recall, after we reconnected the printer to the network and sent a job to it, everything worked fine. Within a couple of hours, he was out of a job, and I was scrambling to secure our network in case he’d left any surprises behind.

So if you want to hire someone with a hacker mentality, check references, and make sure multiple people are willing to vouch for the candidate’s honesty.

But if I have any bone to pick with that article, it’s the idea that you only need one person like this. If you can afford a couple of them with slightly different and complementary skill sets, you’ll do better. They can work on different projects at the same time, and they’ll challenge each other and force each other to grow.

2 thoughts on “Confessions of a hacker for hire”

I agree, when people say “hire a hacker,” they have one of (at least) three ideas: there’s the hacker definition of “jack of all trades,” a guy that can come up with “outside the box” solutions — scripts, tricks, and hacks, so to speak. Then there’s the hacker “security professional,” a guy that has certifications or knowledge specific to hacking as it pertains to security. And finally there’s the grey or black hat hacker, the kid they caught “breaking in” and trying to turn the tables on him by hiring him.

I currently wear the first two hats. I am my branch’s “outside the box” guy, to the point where people have no qualms about asking me to do seemingly impossible things, mostly because of seemingly impossible things I have delivered in the past. I’m the guy that turned our department’s three-day patch deployment system into an hour long automated one. These guys (and my opinion may be tainted) are invaluable. Looking outside the box and coming up with solutions that managers or other computer people may not have thought of or even understand can save companies tons of time and money. Like you said, these people rarely get to talk at meetings (I’m a good example of why they should not be able to do that). I work in a basement and they keep me away from the general public. When they want something they bribe me with drive space and energy drinks. So far, it’s working.

I am also the second kind of “hacker,” a security guy who runs tools and does testing and generates reports. As you either know or suspect it’s nowhere near as romantic as it sounds. At the end of the day it’s all numbers and data. I have people constantly worried that I’m looking at their e-mail or their data — trust me, I have so much to do, I couldn’t care less about viewing people’s personal photographs or e-mails. These guys are also a great asset to businesses. Farming out security scans can be almost as expensive as having someone do them in house, and there’s a certain amount of security in knowning that your findings are being kept internal. Plus, the better an auditor understands your environment, the better. You just have to make sure that the results you end up with are impartial.

We’ve all heard romantic stories of the third kind of hacker, the bad kid turned good, the kid who got caught with his hand in the cookie jar and, instead of prosecuted, given a job. This is a great concept in fiction, but in reality, it rarely works out. Human beings (generally speaking) rarely find a new wealth of morals by simply being offered a job. Someone that was willing to snoop around without authorization is likely to do it after you hire them, too. Like you found out, these kinds of hires rarely turn out to be very trustworthy. Chances are before long they’ll be digging into your business, or into someone else’s using your resources.

I noticed a pattern. An interesting pattern. You and I weren’t CS majors. The other two guys I had in mind when writing this (good, ethical, high achieving guys) weren’t either. I think one went back and got a CS degree.

Seems to me that having one, maybe two computer-talented non-CS majors on staff is an asset. It’s one reason I refuse to apologize for not having a CS degree.