That needle in the haystack of useful big data may be smaller than we thought

New research analyzing the log data churned out by applications developed on the Heroku platform as a service shows just how little of that data is actually useful to developers or devops personnel running those applications.

Out of 22 billion log events across 6,000 Heroku applications, just 0.18 percent held information that a developer (or a devops pro) would actually need to know to prevent a failure or boost performance, according to Logentries. Taken another way, 99.82 percent of that data is the haystack of non-useful stuff.

Of course, Logentries is publicizing this because it’s in the business of helping people winnow out that 0.18 percent without having to sift through the rest, but still, it’s interesting.