Bloomberg Vault Dives Into Dark Data

Bloomberg's enterprise information management service launched File Analytics to help companies manage and unlock the potential of unstructured data.

Bloomberg's enterprise information management service, Bloomberg Vault, is expanding its capabilities into "dark data" with the launch of its File Analytics product.

The new hybrid cloud service enables enterprises to manage dark data, which Bloomberg defines as unstructured data that is difficult to identify, categorize, track, and manage. Dark data can include messaging, social, files, documents, and voice.

"This is a way to help the industry solve its challenge around unstructured data," said Harald Collet, global head of Bloomberg Vault, in a recent interview at Bloomberg's New York City headquarters. "Eighty percent of [unstructured] data is generated by humans in the form of documents, emails, and recorded phone calls and is typically harder for employees to manage." With the explosion of data, firms have a file drawer full of documents, contracts, and shared information. One organization working with Bloomberg had more than 1 pedabyte of data. "This represents discovery risk."

A 2011 IDC study (sponsored by EMC) found that the world's digital data (which includes measurement files, video, music files, and so on) is doubling every two years. Financial services companies are grappling with a big data problem sparked by the adoption of new regulations and compliance requirements.

Bloomberg Vault File Analytics is designed to help firms address these data challenges and derive more value from their data by enabling them to manage, analyze, and control large amounts of enterprise information, Collet said. "What people want is analytics."

Polygon Global Partners, a global investment firm, tested Bloomberg Vault File Analytics over the past several months. "Like most businesses in our space, we handle large volumes of unstructured data," Polygon's technology chief, Neil Roberts, said in a press release. "Bloomberg Vault File Analytics is a mechanism to better extract key insight and analysis from this data while also allowing us to responsibly maintain the correct audit trail for compliance and recording keeping."

Bloomberg said in the release that File Analytics helps organizations mitigate risk and alleviate the concerns of compliance, legal, or risk management. Legal and compliance professionals can use it for policy-based record retention, regulatory investigations, e-discovery collection, and deletion of corporate data based on best-practices. For example, firms can run policies off File Analytics for social media and voice data.

Trade reconstruction Two weeks ago, Bloomberg Vault announced the results of a study on the challenges of meeting the CFTC trade reconstruction requirements under the Dodd-Frank Act for swap dealers and major swap participants. The informal survey was conducted at a roundtable that Bloomberg Vault and Compliance Risk Concepts hosted at Bloomberg headquarters.

Only 7% of the compliance officers attending felt confident they were ready for meet the CFTC's requirements. Trade reconstruction is another type of information analytics that File Analytics can handle for firms facing complex recordkeeping requirements.

Firms must reconstruct a trade within 72 hours, while the information cuts across different transaction systems, Collet said. File Analytics will help firms reconstruct their swap trades, including searching for and tagging unstructured data with trade and legal entity identifiers, which 100% of survey participants said was the hardest part of pre-trade workflow.

IT executives can leverage the technology for file reporting, intelligent data migration, and more cost-efficient storage management across their enterprise data and file shares, according to the release.

In terms of delivering the service, Bloomberg maintains that it's one of the few firms solving this problem with cloud technology.

"We're going to be working with our clients on how to manage this data using a hybrid cloud approach," Collet said. Bloomberg Vault runs on the Bloomberg cloud, which is "a unique, secured private cloud." The company operates more than 120 data center sites, and it has the scalability and advanced security to support global enterprises.

With the adoption of many new regulations, Bloomberg Vault can provide these services at a lower cost, and with better service-level agreements, than IT departments, Collet said.

Interesting that Bloomberg chose to use "dark data" for unstructured data. Although 'dark data' is a common term for unstructured data, in the financial services space some users might think that 'dark data' is information and stats on the dark pools (volumes, matches, and so on).

LOL! I see where you're heading given the recent news. There is no connection to dark pools here. Dark data is meant to convey data that's hidden in a file drawer or a corporate file or repository. Firms may know the data exists, but once it's in a drawer they don't have easy access to it and it may not be tagged. Bloomberg is helping firms to search, organize and analyze this unstructured (dark) data.

Ha, I'm with Greg on this. To my mind, "Dark" is also synonomous with "black-market" or "dirty" data, something firms would rather not have discovered in their files. (Dark pools sound way more ominous to me than they should). Maybe they should call it shadow data, or forgotten data... No? Anyone? Anyone?

The word "dark" automatically conjures up a sinister side to whatever noun is being modified. We see this with dark pools and despite industry attempts to rename the them as "non-displayed liquidity pools," it didn't work. Dark pools is a much catchier term though it unfortunately seems to attract misbehavior. Now we have 'dark data' to describe the data that could be forgotten in a file or corporate repository, but most likely is unstructured and diverse. I'm sure marketers could try different terms, but dark data has a ring to it.

LOL Can you imagine how different the industy's attraaction would be to that long-winded name. I don't see much traction on "Bloomberg Vault Dives into Non-displayed Liquidity Pools" Not going to happen.

The ability to deep dive into dark data is going to have a big impact on these firms's abilities to respond to regulators, but also to build models and spot inefficiencies in their record keepings. File Analytics is especially appealing that it can help identify and manage social data and voice data- which is clearly rapidly becoming unmanageable.

Cutting across all the data types for pre-trade workflows is clearly no picnic, but companies are not exactly making moves to limit the types of data-types in their systems. When regulators come knocking with the 72 hour deadline I think Bloomberg will see a strong response from industry participants looking for pragmatic solutions.