Intel Launches Its Own Apache Hadoop Distribution

Intel is leveraging its hardware assets in order to speed its new Hadoop distribution.

It’s no secret that Apache Hadoop has caught on in a big way among companies large and small. The open-source framework specializes in running data applications on large hardware clusters, making it a particular favorite among firms such as Facebook and IBM with a lot of backend infrastructure (and a whole ton of data) to manage. Studies indicate that Hadoop’s popularity will only increase over the next few years.

So it’d be hard to blame Intel for jumping into this particular arena. The chipmaker has produced its own distribution for Apache Hadoop, apparently built “from the silicon up” to efficiently access and crunch massive datasets. The Intel Distribution for Apache Hadoop software includes Intel Manager for Apache Hadoop software. The latter features Intel Active Tuner for Apache Hadoop, which supposedly configures the platform for optimal performance.

The distribution takes advantage of Intel’s work in hardware, backed by the Intel Advanced Encryption Standard (AES) Instructions (Intel AES-NI) in the Intel Xeon processor. Intel also claims that a specialized Hadoop distribution riding on its hardware can analyze data at superior speeds—namely, one terabyte of data can be processed in seven minutes, versus hours for some other systems.

“People and machines are producing valuable information that could enrich our lives in so many ways, from pinpoint accuracy in predicting severe weather to developing customized treatments for terminal diseases,” Boyd Davis, vice president and general manager of Intel’s Datacenter Software Division, wrote in a Feb. 26 statement.

That interest in Hadoop likely won’t abate anytime soon. Last summer, a report from Market Research Media suggested that the framework’s popularity could continue over the next several years, transforming into a $2.2 billion market by 2018 (other research firms have pegged the market at a somewhat lower valuation—IDC, for example, believes it’ll hit $812.8 million by 2016).

That being said, Hadoop’s open-source nature could hinder those IT vendors seeking to make some cash off proprietary platforms. IDC believes that competition between open-source and commercial platforms may drive vendors to lower product-licensing fees, which would reduce revenues. Whatever happens, Intel definitely wants its piece of the game.

Top Stories

Author

Mark Hachman is a freelance editor at Slashdot Media. Previously, he served as the West Coast news editor for PC Magazine/PCMag.com, where he covered components, new technology, and the larger Web 2.0 companies such as Facebook and Google. Before that, he worked for eWEEK, TechWeb, and ExtremeTech, which he helped launch.