Legal hold is a necessary annoyance for corporations, government bureaucracies and other large organizations. The term refers to the process of preserving all types of data -- including documents, email, voicemails, text messages and even social media posts -- that may be relevant in case of legal action.

But once an enterprise stockpiles what could amount to tens -- or even hundreds -- of millions of relevant pieces of information, how can it sift through the data quickly? After all, efficiency is essential during electronic discovery (e-discovery), the step in civil litigation where opposing parties exchange electronically stored information.

Eager to fill this market niche, several software vendors have developed document management systems designed specifically for e-discovery. These products include Symantec's Clearwell, Recommind's Axcelerate and kCura's Relativity.

"We all have a slightly different spin to it, but fundamentally we're doing the same thing," said Andrew Sieja, kCura founder and CEO, in a phone interview with InformationWeek.

According to Sieja, kCura's customers include top U.S. law firms, large corporations and government agencies such as the U.S. Department of Justice. "But our biggest segment comes from consulting firms that help corporations and law firms through large litigation projects," said Sieja.

KCura is currently managing about 27,000 active cases, and its software has roughly 73,000 users. In total, the company's software is managing close to 18 billion files, according to Sieja. "Probably a lot of the really messed-up stuff happening in the world right now, from a litigation standpoint, is being managed in our software," Sieja added.

KCura and its competitors use a machine-learning technology called predictive coding, which uses limited human input to enable a computer to "predict" how documents should be classified, according to attorney Matthew Nelson's Predictive Coding for Dummies book.

"Let's say you have a corpus of data with about a million documents," said Sieja. "Before you start your predictive coding project, you'll identify how accurate you want the review to be." Sieja offered this example: "I want to be 95% confident that we're going to be 100% accurate with a 3% margin of error." Based on these thresholds, the next step is to serve a statistical sample of the data to one or more expert attorneys, who review the documents and decide which are relevant to discovery requests.

"It's usually around the area of 1,500 docs," said Sieja. "Then we'll submit those decisions into the system, and based on their conceptual makeup we'll automatically classify all the other documents."

The process doesn't end there, however. A sample of the classified information is sent back to the experts, who review the material. If the computer-generated predictions don't match the accuracy target, the process is repeated again. "They'll continue this process until we're within the statistical range," said Sieja.

For corporations, governments and other large organizations, document management systems can expedite the e-discovery process, and save huge sums of money in document-review costs.

The rise of big data presents a challenge, however, and not just due to growing stockpiles of information. "In the world of e-discovery, a 100 million-document record set is very big for us. That's a lot of data for us to make sense of from an e-discovery standpoint," Sieja said.

Unstructured data poses another problem. "People are collecting data from phones now," said Sieja. And that's in addition to emails, text messages, Twitter and Facebook posts, and information from enterprise collaboration systems like SharePoint.

"If you look at where we're taking our product development roadmap, it's all about scalability and handling more data," Sieja noted.

He added: "A lot of the stuff that we do today is very reactive. More and more of these corporations are going to get more proactive in how they think about e-discovery."

Attend Interop Las Vegas May 6-10 and learn the emerging trends in information risk management and security. Use Priority Code MPIWK by March 22 to save an additional $200 off the early bird discount on All Access and Conference Passes. Join us in Las Vegas for access to 125+ workshops and conference classes, 300+ exhibiting companies, and the latest technology. Register today!

The rising volumes and types of data residing within a corporation can stymie efforts to track, manage and control important and sensitive data that might be subject to litigation or investigations. When it comes to legal review, it has become a necessity to reduce volumes due to time, resource and cost constraints. Certainly, when applied with the proper processes and human expertise, technology-assisted review (often called predictive coding) can be hugely beneficial in quickly and cost-effectively winnowing down stroves of data to a smaller, more manageable population that is valuable to the case. That said, internal teams are simultaneously focused on finding ways, often with the assistance of third-party data management and business process experts, to proactively manage data from an information governance perspective GÇô including the design, implementation and monitoring of a robust information retention and defensible data deletion program. Once in place, data volumes subject to preservation may be more manageable at the ouset. And, with the appropriate search and analytics technology, legal teams can routinely cull down this data before the review begins GÇô keeping potentially huge costs and risks at bay, and moving through document reviews at a fraction of the time and cost of traditional manual methods, without sacrificing accuracy.

IT’s tried for years to simplify data analytics and business intelligence efforts. Have visual analysis tools and Hadoop and NoSQL databases helped? Respondents to our 2014 InformationWeek Analytics, Business Intelligence, and Information Management Survey have a mixed outlook.