"You want me to what?" Hugo asked for the third time during our phone conversation. "Bring him here. Scrap and I just wrapped the technical side of this case up. It's going to be up to Dunnit to wrap up the human side." I said. "Chief, he doesn't want to come back. Well, unless..." Hugo said, drifting off into a whisper. "Unless what?" I demanded. "If you can get Kiltlifter to apologize to Dunnit, he might just come back. I know for a fact though that he's not going to magically appear at the university before then." Hugo replied. "Okay, keep an eye on him and tell his sister what has happened. Maybe she can talk some sense into him." I hypothesized. "And tell her I said 'hi'!" Scrap barked from a location behind me. I turned my attention to Kiltlifter's workstation. It was identical to Dunnit's, and sat proudly on Kiltlifter's desk amongst a pile of neatly organized papers. "Shall I power it up, boss?" Scrap asked as he sat down in Kiltlifter's chair. "Exactly as you did before when you two were looking for Dunnit's files on the SAN." I said. Scrap pressed the power button on the workstation and the power supply fans spun to life. In a few moments, the monitor glowed with the familiar boot messages of Gentoo Linux and eventually a login prompt. "Here goes nothing!" Scrap joked as he logged in with Kiltlifter's credentials. "Great. Look at /mnt/storage please." I asked. Scrap changed the current directory to /mnt/storage and performed a 'ls' command to list any available files. There were none. "See. This is exactly what happened last time, man." Scrap whined. "What exactly am I supposed to be seeing, Obi Wan Chimp-nobi?" "Open up /var/log/lastlog. Grep for '/dev/s??' and tell me what you see." I instructed. Scrap performed the command and was greeted with a list of every hard drive device in the SAN. "Okay, still not seeing it man. Maybe you should use your 'force powers' and stir me up a latte and a blonde named 'Bambi'?" Scrap cracked. "Seriously, what am I supposed to be seeing?" "Nothing yet, my impatient marsupial. Pop in the thumb drive that I copied Dunnit's /var/log directory to. Do a 'diff' of Kiltlifter's /var/log/lastlog and Dunnit's /var/log/lastlog. Now what do you see?" I asked. Scrap's fingers whipped around on the keyboard and in a matter of seconds he was looking at the output of the 'diff' command. "Holy needle in a haystack. No way! No fricking way!" Scrap cried, nearly falling out of the chair with laughter. "Now you see it! Although both machines are identical in every way aside from the /etc/fstab entries, for some strange reason that can only be attributed to the Linux kernel and/or modules, Kiltlifter's machine assigns /dev/sda to a different, empty volume on the SAN!" I exclaimed. Scrap was laughing so hard he was turning red. "You mean to tell me that 'X' out of 10 times that Kiltlifter boots his workstation, his /mnt/storage partition is going to be pointed at the wrong volume? And you're telling me that because of this, ol' Kiltlifter assumed that Dunnit had wiped the SAN volume and fled town? Oh man, this is rich!!!" Scrap wailed. I grinned. "That's exactly it. It didn't hit me until I paid attention to the boot up kernel messages. As joyful of an occasion as this may be, we only have half of the equation solved. Now Hugo needs to get Dunnit back here and they need to do some serious talking." "Yeah, yeah. So when do we get to tell Kiltlifter? I can't wait to see the look on his face!" Scrap said. "No time better than the present." I said as I picked up my cell phone and dialed Kiltlifter's cell phone. Conclusion: Scrap and I spent a good deal of time with Kiltlifter explaining the technical details behind his 'snafu' with the SAN. After the facts were laid out, Kiltlifter agreed to do what was necessary to make Dunnit feel comfortable enough to return to the university. Hugo played mediator between Kiltlifter and Dunnit and talked through most of Dunnit's issues. After consulting with an attorney, Dunnit parted ways with the university and is currently exploring private research on what would have been 'project charlie'. Projects 'alpha' and 'beta' are currently distributed by a large pharmaceutical company in the United States and Europe and play an active role in battling several types of cancers. This case is a perfect example of how lack of technical expertise and poor investigative work on the client's part turned into something much larger than needed. Scrap and I recommended to Dunnit that they work with Haystack to resolve the volume mounting problems and explore an adequate backup solution. We also performed a brief risk analysis on the network and advised Kiltlifter that the network access controls were quite lacking. One of my earliest mentors used to say "No matter what the subject matter, the persons involved, the crime committed, or the resolution found, you can never predict human nature. It's the most random element in the universe, and it was conveniently left off of the periodic table." He can say that again.

Some name

Security Monkey is an expert investigator of computer crime for businesses small and large. His background includes military and federal experience in intelligence, electronic countermeasures, and signal and information interception.

"I'm not the world's greatest expert on Linux but I've seen flaky drivers and bad cables cause this type of problem. Manufacturers can be your source as well. For example, one of the big pc providers, name starts with a D, has really excellent hardware and driver support but they like to switch hardware vendors even for the same model device. You can't always guarantee that 20 of the same model workstation will have the exact same HW. "