Role in IT decision-making process:Align Business & IT GoalsCreate IT StrategyDetermine IT NeedsManage Vendor RelationshipsEvaluate/Specify Brands or VendorsOther RoleAuthorize PurchasesNot Involved

Work Phone:

Company:

Company Size:

Industry:

Street Address

City:

Zip/postal code

State/Province:

Country:

Occasionally, we send subscribers special offers from select partners. Would you like to receive these special partner offers via e-mail?YesNo

Your registration with Eweek will include the following free email newsletter(s):News & Views

By submitting your wireless number, you agree that eWEEK, its related properties, and vendor partners providing content you view may contact you using contact center technology. Your consent is not required to view content or use site features.

By clicking on the "Register" button below, I agree that I have carefully read the Terms of Service and the Privacy Policy and I agree to be legally bound by all such terms.

New Apache Project 'Drill' Aims to Speed Up Hadoop Queries

Led by Hadoop vendor MapR Technologies, the open-source effort will seek innovative ways to push Hadoop data queries through more quickly for users.

Finding much faster ways to complete Hadoop queries for enterprise users is the aim of "Drill," the latest open-source project being undertaken by the Apache Software Foundation.

Drill has been established as an Apache Incubator Project, opening its continued development up to software engineers around the world, according to Tomer Shiran, director of product management for Hadoop vendor, MapR Technologies, which is one of the backers of the Apache Drill project.

"We've spent quite a few months talking to lots of organizations and potential users of Drill and to our customer base as well," said Shiran, who is a founding member of the Drill project. "We wanted to put this out there as an open-source project, rather than just keep it within MapR for our use alone."

Further reading

"With Drill, you'll be able to get really fast responses," he said. Users will be able to get responses within one second, which is a key difference from other tools that are available today, he added.

As it presently works as it was designed, Hadoop does batch processing of large data sets. Drill will improve on that method by doing "interactive analysis" that can find the required answers in the data more quickly, said Shiran. "Interactive analysis is much faster than batch processing."

The need for tools like Drill has been inspired by always-increasing user requirements, he said. "People have been doing queries in Hadoop, but since it doesn't return answers to you within a few seconds, it has limitations."

Using Drill, users will be able to do ad hoc analysis and get faster responses, whether they are seeking anomalies, data trends or even network intrusions, according to Shiran. "With all of those things, you're going to have to get a pretty fast response or by the time you do figure it out, it's going to be old news."

The nascent Drill open-source project is currently in development and includes a variety of companies and individuals who are working on it right now. "A broad-based effort will be working on this," said Shiran. "There's quite a few people actively developing on the project now, so I don't think it will be a long time before we have an early version released."

Drill was inspired by Google's Dremel project, which helps Google perform data analyses on its huge data sets such as analyzing crawled Web documents, tracking install data for applications on the Android Market, analyzing spam, analyzing test results on Google's distributed build system and more, according to Shiran.

By developing Drill as an Apache open-source project, organizers will be able to establish Drill's own APIs and establish a flexible and robust architecture that will support a broad range of data sources, data formats and query languages, according to the group.

MapR offers two versions of its Hadoop products: MapR M3, which is free; and MapR M5, which is a commercial version of the product with advanced features, including high availability, the ability to make data snapshots and do mirroring of datasets, and 24/7 support.