This investigation into the remote spying allegedly being conducted against students at Lower Merion represents an attempt to find proof of spying and a look into the toolchain used to accomplish spying. Taking a look at the LMSD Staff List, Mike Perbix is listed as a Network Tech at LMSD. Mr. Perbix has a large online web forum footprint as well as a personal blog, and a lot of his posts, attributed to his role at Lower Merion, provide insight into the tools, methods, and capabilities deployed against students at LMSD. Of the three network techs employed at LMSD, Mr. Perbix appears to have been the mastermind behind a massive, highly effective digital panopticon.

PanoMasterMind

The primary piece of evidence, already being reported on by a Fox affiliate, is this amazing promotional webcast for a remote monitoring product named LANRev. In it, Mike Perbix identifies himself as a high school network tech, and then speaks at length about using the track-and-monitor features of LanRev to take surreptitious remote pictures through a high school laptop webcam. A note of particular pride is evident in his voice when he talks about finding a way outside of LANRev to enable "curtain mode", a special remote administration mode that makes remote control of a laptop invisible to the victim. Listen at 35:47, when he says:"you're controlling someone's machine, you don't want them to know what you're doing" -Mike Perbix

It isn't until 37 minutes into the video till Perbix begins talking about the Theft Tracking feature, which causes the laptop to go into a mode where it beacons its location and silent webcam screenshots out to an Internet server controlled by the school.Click to watch an excerpt of Mike Perbix's spycast

The beacon feature appears to have been one of the primary methods for remote spying, however, network footprints abound over the details and architecture of the remote administration effort. In this post, Perbix discusses methods for remotely resetting the firmware lockout used to prevent jailbreaking of student laptops. A jailbreak would have allowed students to monitor their own webcam to determine if administrators were truly taking pictures or if, as the school administration claimed, the blinking webcams were just "a glitch."

Perbix also maintains a prolific blog, where in this blog post he describes using the remote monitoring feature to locate a stolen laptop:

"As a prime example, we initially attempted to recover a stolen laptop that reported back to us it's internet address and DNS name. The police went to the house and were befuddled to find out the people we knew had the laptop was not the family that lived there...well, we eventually found out that they were the neighboring house and were borrowing the unsecured WI-FI."

What's the purpose of shutting down a camera for the user of the laptop but still making it available to network administrators? Ask yourself: if you wanted to convince someone that a webcam blinking was a glitch, would disabling the cameras help make your case?

We Found the Glitch, Mrs. Buttle

The truly amazing part of this story is what's coming out from comments from the students themselves. Some of the interesting points:

Possession of a monitored Macbook was required for classes

Possession of an unmonitored personal computer was forbidden and would be confiscated

Disabling the camera was impossible

Jailbreaking a school laptop in order to secure it or monitor it against intrusion was an offense which merited expulsion

When I spoke at MIT about the wealth of electronic evidence I came across regarding Chinese gymnasts, I used the phrase "compulsory transparency". I never thought I would be using the phrase to describe America, especially so soon, but that appears to be exactly the case. On a familiar note, the authorities are denying everything. As one reads comments on this story, a consistent story begins to emerge:

Browse as many web forums as you like, the comments above are highly representative. Students were told green webcam activation lights going off at home were a glitch, were required to use a jailed computer, were threatened with expulsion if they attempted to jailbreak the computer to find the truth, and were not allowed to use computers they controlled.

Inside LANRev

With some of my colleagues, I began a reverse engineering effort against LANRev in order to determine the nature of the threat and possible countermeasures. Some of the things we found at first left us aghast as security pros: the spyware "client" (they call it an agent) binds to the server permanently without using authentication or key distribution. Find an unbound agent on your network with Bonjour, click on it, you own it. The server software, with an externally facing Internet port... runs as root. I'm not kidding. For those unfamiliar with the principle of least privilege- this is an indicator of a highly unskilled design. Unfortunately, when we got down to basic forensics, LANRev appears to cover its tracks well. Here's a screenshot of the server application monitoring a tracked host:

Tracking intervals available at the top; screenshots and webcam shots in the lower right pane. No webcam shot is visible here as a webcam was not connected during testing

In order to spy on my computer, I had to mark it for spying. The icon for spying is a detective hat and a magnifying glass; very Sherlock Holmes

Once I had the agent installed, I used dtrace to monitor its activity as it hung around and spied on my system. The log below is an edited trace of the agents activity during a spy interval. It uses a fixed dump point, /tmp/Image, as its save file before uploading to the server, sadly this is wiped. Only a full forensics scan which picks up deleted files will have a chance of picking up the history of the spying on a particular computer. On laptops with a webcam, a second fixed save point, /tmp/Image1, is used to save the webcam pic.

For the technically inclined, I've highlighted some of the key points, use of the system screengrabber, the use of RawCamera, the fixed save point, etc. We're still working on our technical writeup of this software and hope to update soon.

During our testing, we infected a laptop with LANRev, then closed the lid, hoping to activate the LANRev feature which takes a webcam picture when the computer wakes. As my colleague Aaron opened the lid of his Mac, the green webcam light flickered, ever so briefly. It wasn't a glitch. It was a highly sophisticated remote spy in his system. And even though he was in control, the effect was still very creepy.

Here's one last capture from the Windows version of the administration console, showing a forced remote webcam snapshot. We've pixellated this, but rest assured the real thing looks very detailed

In other news on the case, subpoenas have been issued, the FBI is on the case, the candy in question has been caught red-fingered, and some enterprising chap is ready to cash in with a t-shirt. Doug Muth's hands on screenshots provide the best first hand encounter with the client end of the spyware in question. What amazes me most is that the family and lawyer filing the suit appear to have done no digital forensics going in, and no enterprising student hacker ever jailbroke a laptop and proved this was going on. The greatest threat to this investigation now is the possibility that the highly trained technical staff at LMSD could issue a LANRev script to wipe digital forensic evidence off all the laptops. This is why it is imperative for affected parents to have the hard drive removed from their children's laptops and digitally imaged before the laptop is connected to a network. With enough persistence, and enough luck, we may eventually learn the truth.

-stryde.hax

update 2/23/2010 6:00pm

If you haven't already, you must watch this PBS Documentary - How Google Saved a School. At five minutes in, you can see all these same features in use, in a school setting, by a principal. Remote surreptitious observation. Remote camera use. All used by a principal to observe kids and make sure they're working. There are a lot of school districts, administrators, IT professionals, and security professionals who see nothing wrong with this documentary. They see remote administration software in use in this way and they don't think it's wrong, and they don't think it's spyware. Some of them even believe that the extension of this functionality into the home doesn't make it spyware, or even wrong. But this is my personal blog, and it's my personal opinion that they're wrong. As an expecting parent, I don't ever want my kids on the business end of Remote Desktop Curtain Mode, even at school. I'm a security professional, and a big part of my education and my professional development was tinkering and tearing apart computer systems to gain understanding, learn how they work, and change their use. I believe that computer security is knowledge in practice; it's using your knowledge to protect yourself. These kids are learning that security is something that happens to you. That's backwards. DARPA thinks we're not raising a generation with applicable security skills. I think they're right; I think this is a recipe for the next generation of phishing victims. I'd like to see a school system where a kid can bring in x64 Ubuntu or Haiku OS that he secured him/herself. I'd like to see a school system where kids teach each other how to defend against remote webcam use. Instead, we've got kids who can't run Terminal. Not my kids.update 2/23/2010 4:12pm

A note for anyone wishing to contact me privately: if you'd like me to write back, please leave a return email. My email is still stryde dot blog at gmail dot com. It's Not Spyware!!!I've received a lot of positive feedback about this entry; however, if there's one consistent complaint amongst my detractors, it's my classification of LANRev as spyware. So here is my response. Confusing remote admin software with spyware has a long history stretching back to Cult of the Dead Cow's first Bo2k release. I'm not as funny as them so I don't even try. It's true however that remote administration tools and spyware exist on the same spectrum, just ask the guys at Spectresoft. Spyware authors and remote admin authors often have to solve the same problems, like bypassing OS protections and getting around antivirus. It's a transition that's easily made. So where's the dividing line? The line is basically in how its used. Remote admin usually solves constructive tasks, like remote patch management, inventory location tracking, remote software installation. And sometimes it means screensharing in order to solve problems. I personally have sat at home as a network tech worked on my corporate laptop over a VPN. No problem. My personal opinion is this: when you see a piece of software with dedicated functionality for taking webcam screenshots surreptitiously and removing the evidence on disk, to me that's crossed the line into spyware. I'm certain that others in the industry will disagree with me. That's fine; let's have the debate. I don't mind losing a technical argument, as long as it's on merit.

update 2/23/2010 11:28am

My colleague Aaron pointed out to me today that the reason LANRev is using the raw camera device is that Apple implemented security measures to prevent remote activation of the webcam in OSX. LANRev was designed to bypass this security measure. Those who disagree with my spyware assessment, ask yourself, "what kind of software bypasses OS security measures?" On the topic of whether or not we yet have proof of illegal use, I would ask you to listen carefully to the webcast, and listen for the word "house" at 1:28. Listen for "yes we have used it."update 2/23/2010 10:00am

I've removed Mr. Perbix's picture from my blog. I try very hard to stick to verifiable facts when I write here; this blog post is made up references to primary documents that show a verifiable pattern of action. But I feel that some readers are getting carried away. Myself and Aaron Rhodes spent hours reading forum posts, messages, and communications from Mike Perbix, his "digital shadow". The impression we both got was of a man who was charged with enormous responsibility, worked very hard, was very adept, and was fanatical about protecting kids and the assets he was charged with managing. I don't have all the facts yet, but the impression I got was of someone who was trying to build a state of the art capability and revelled in the promise of technology. If I had to put my finger on what when wrong here, I would say that someone cared too much. Personally I'm much more interested in who this capability was distributed to, and its persistent pattern of access, than I am in the person who built it. If you're reading this, please, let us not participate in a rush to judgement especially against a guy who worked this hard. Yes, he built the capability. Yes it was used. But if it was abused or simply misguided, that remains to be proven. I for one reserve judgement. For now, what bothers me most is this: When an organ of the State (in this case, a school) builds a system to conduct a search by activating webcams off of school grounds, the only way to determine if the ensuing search will be unreasonable or illegal is to conduct the search. The thought process behind that is unfathomable to me, no matter how much I read about it.update 2/22/2010 8:30pm

I've created a network footprinting capability for parents, students, anyone who may be concerned that they are infected with the LANRev agent. The capability is documented in my next blog entry. One piece of feedback I continue to get is speculation on what can be seen in a packet sniffer. The answer for now is: not much. A block cipher and compression are in use in serial. It's a tough problem; we're working on it.update 2/22/2010 5:30pm

I've watched the 50 minute screencast repeatedly, where Perbix describes his use of this feature outside of school grounds repeatedly during a conversation with Absolute Software employees. They were enthusiastic... now they're throwing LMSD under the bus? I believe this can best be described as intense PR spin. It also completely confirms what I've asserted here, that LANRev was the implant of choice for this school.

stryde.hax has done a great job here. Absolute power corrupts absolutely. Never underestimate the creativity of public education administrators (government workers) to extend and enforce their control over their charges. “Mandatory” public education provides a constant and unending source of captive subjects.

I’ve seen some interesting accusations about the family who brought this suit against the district. Regardless of their track record or their motivations it is clear they were the recipients of public educations obfuscation and denial strategy. The only language the system recognizes is a good lawsuit and even then that is tenuous. They believe they are omnipotent and above the law and never hesitate to act on it. Most times there is no recourse but to sell your house and move away.

Whether due to purposeful action on the district’s part to violate privacy rights or due to pure stupidity, they are in a heap of trouble. Let them be an example and serve as a warning to others. Left unchecked government run education is frequently a danger to the well being of children and their families. “Children First” is the myth they hide behind. Don’t listen to what they say, watch what they do. . . very carefully.

I as a Former US Navy Intel professional find it strange that before we(the DOD) did ANY recon or recon type activities w/in the US(we included Canada as well). We had to obtain a “Proper Use” statement that satisfied Intel Oversight Requirements undoubtedly signed of by legions of JAG Lawyers but this little POS “School” distinct could do this...the mind boggles.

They’ll claim ignorance of the law. That will be their defense, and they’ll think that’s sufficient. After all, they are public school teachers and administrators. They are only checking up on the children for their own good, since parents are too ignorant, shiftless, and lazy to do so, even in a school district where the parents are probably much better educated than they are.

I bet a lot of people are losing sleep over this. There should be “zero tolerance” for this type of criminality. We aren’t talking about plastic knives in lunchboxes or drawings of guns or fake drugs. We are talking about real criminal behavior and ignorance is no excuse.

ANOTHER REASON TO HOMESCHOOL

This ping list is for the other articles of interest to homeschoolers about education and public school. This can occasionally be a fairly high volume list. Articles pinged to the Another Reason to Homeschool List will be given the keyword of ARTH. (If I remember. If I forget, please feel free to add it yourself)

The main Homeschool Ping List handles the homeschool-specific articles. I hold both the Homeschool Ping List and the Another Reason to Homeschool Ping list. Please freepmail me to let me know if you would like to be added to or removed from either list, or both.

16
posted on 02/24/2010 9:35:37 AM PST
by metmom
(Welfare was never meant to be a career choice.)

They will also claim that the parents/student are crazy/menace/troublemakers/unreasonable. The first thing they do is make you think you’re the crazy. Even many, many years later it is clear to me that process must be taught in Education Administration 101. Rules of engagement: deny, obfuscate, meet complaints with steely silence, claim there’s nothing they can do, blame the victim and then offer pity. Once parents are wild eyed and crazed with frustration and anger, dismiss them as wackos. Case closed.

For 2300+ PA state felonies, and 2300+ federal felonies, for all the members of the school administration who ordered, monitored, and used the spy capabilities, and for the technical people who installed the hardware/software, ...

... the temp fixes for such as this is usually called incarceration pending trial and sentencing.

There have been a lot of people on these threads trying to say “it probably didn’t happen the way the lawsuit says” or “these kids have no rights anyway”. Based on this article and my own knowledge of the field (M.S. in Computer Science) I will say right now that the charges in the lawsuit are correct, the school district has been spying on students in and outside of school. They may or may not have done this only for theft prevention purposes but this is NOT power that a school administrator should have.

Especially notice the bit in the story about the admin bragging about sending cops to hunt down a stolen laptop, and how they went to the wrong house because the real perps were stealing wireless. First off, how would you like to be the innocent people? Second, if the school sent the cops to the wrong address it means they were using more than just the camera to track the laptop. Just the camera would have sent them to the right place. They were actively figuring out what network the computer was on and WHO OWNED THAT NETWORK. Anyone see issues with that? It’s a good thing in that case that they did use the camera too or I bet the cops would have arrested the innocent homeowners. After all, the school apparently had enough evidence to get cops out there (and we’ll assume a warrant but who knows...)

Yeah? How about disabling the rest of it - like the software that let them figure out the address of the home with the wireless network that a missing computer was connected to? Might need more than a sticky note there.

What should be the real wake up call is the arrogance of the school district in dictating this whole scenario, down to what the kids can and cannot do off school premises, and the fact that the parents and students just accepted it.

36
posted on 02/24/2010 11:25:36 AM PST
by metmom
(Welfare was never meant to be a career choice.)

What should be the real wake up call is the arrogance of the school district in dictating this whole scenario, down to what the kids can and cannot do off school premises, and the fact that the parents and students just accepted it.

Howard vs. Colonial School District is the Delaware court ruling that started it all.

In a way I feel for the kids because the adults should have taken the lead on making sure the computer was safe. But, then I would argue the kids and their parents had reasonable expectations to privacy which were callously disregarded by the school and the tech. I’m just glad we homeschool. I don’t need another reason to be paranoid about the safety of my kids...

She later added that, in more than a decade as assistant vice principal, she had "never disciplined a student" for actions beyond school property that had no connection to a school-related event, apparently in response to the Robbins lawsuit's allegation the student learned of the webcam surveillance from a school disciplinary action.

I wondered how long it would take for that to happen.

Since it's he said/she said, she can safely deny it and then demand proof that she's lying, knowing that no hard copies exist, or have been destroyed.

However, if nothing else, this did bring to light the fact that these computers had capabilities not honestly disclosed to the students and their families.

48
posted on 02/24/2010 5:38:15 PM PST
by metmom
(Welfare was never meant to be a career choice.)

“However, if nothing else, this did bring to light the fact that these computers had capabilities not honestly disclosed to the students and their families.”

That’s where the school district made their big mistake, hopefully out of ignorance, not malice. Perhaps the principal was unknowingly duped, made a scapegoat by someone (the kid and his parents, the IT Department, someone else?). What a career killer. No matter. The cat’s out of the bag. The computers are compromised, the school’s reputation is in tatters.

That IT department or whatever they call themselves — they’re something else. The FBI will straighten it all out.

Disclaimer:
Opinions posted on Free Republic are those of the individual
posters and do not necessarily represent the opinion of Free Republic or its
management. All materials posted herein are protected by copyright law and the
exemption for fair use of copyrighted works.