Mining email traffic for bad omens

Researchers examined Enron email logs during the 18 months before the shit really hit the fan. Amazingly, just analyzing the number of emails and their paths, without peeking at the content, hinted at the crisis to come. Of course, hindsight is everything. Still, the stiudy, by computer scientists at the Florida Institute of Technology, is provocative. From New Scientist:

After US energy giant Enron collapsed in December 2001, federal investigators obtained records of emails sent by around 150 senior staff during the company's final 18 months. The logs, which record 517,000 emails sent to around 15,000 employees, provide a rare insight into how communication within an organisation changes during stressful times...

(Ronaldo) Menezes says he expected communication networks to change during moments of crisis. Yet the researchers found that the biggest changes actually happened around a month before. For example, the number of active email cliques, defined as groups in which every member has had direct email contact with every other member, jumped from 100 to almost 800 around a month before the December 2001 collapse. Messages were also increasingly exchanged within these groups and not shared with other employees.

Menezes thinks he and (Ben) Collingsworth may have identified a characteristic change that occurs as stress builds within a company: employees start talking directly to people they feel comfortable with, and stop sharing information more widely.

8 Responses to “Mining email traffic for bad omens”

The only problem is that by the time this behavior starts to be obvious, it’s far too late to do anything about the underlying problem. By the time even the peons know, the shit is already on its way to its appointment with the fan.

Still, assuming that a company like Enron was nice enough to provide you with its entire email archive for analysis, it might give you enough time to sell off your own holdings.

A simpler check might be to monitor the email output of senior management for a sudden fall-off. When they stop putting anything in writing, the end is nigh.

Back in the days before the Information Super Highway, Old School spies used to have to communicate by radio. Even if you couldn’t break their codes and cyphers you could tell a lot about what they were up to by analysing the traffic patterns in their broadcasts. The RCMP pioneered the statistical analysis methodologies for doing this in the Cold War.

someone who finesses custom malware into corporations so they know exactly when to short their stocks is a pretty nifty character for a cyberpunk story though. Hopefully, there’ll be someone doing it for real, by the time stocks are worth anything again that is.

@1: I think it’s also a case of people being unsure of the current environment, and so only sharing new information about potential problems with their closest group of peers, both to gain an upper-hand on the less-informed, and to prevent recriminations if you are discussing problems you see.

Anyway, my question is, what is the value of this? I assume that investors/SEC can’t be looking at this data in real-time, so it won’t be helping them. As for the company self-monitoring, it seems clear from this that the people in the company probably did know that things were on the rocks — hence their observed reaction.

Actually, I take that back. Maybe this could be good for self-monitoring. Everyone can only see within their own limited range of vision. Engineers are emailing each other about how recent cuts have made making deadlines impossible. Sales are sending emails to each other about how no one is buying. All these people are aware of their own problems, and maybe no one is in the position to look on this and see that everything is interconnected and going down the tubes.