WEBINAR:Available On-Demand

On a regular basis we take a look at what has been happening in the open source community and make a list of noteworthy new projects that have debuted in the last year or two. As always, the open source community is bursting with new projects. For this list, we’ve focused on two growing categories, Big Data and DevOps.

Analytics and Big Data

Data Torrent has been around a while, but it first open sourced its Core RTS technology in June of this year. It claims to be "the industry's only open source enterprise-grade unified stream and batch platform." It comes in community, standard and enterprise versions. Operating System: Linux

Created by Netflix, Genie allows IT administrators to manage Hadoop jobs running on cloud computing services. Netflix uses it to run many thousands of Hadoop jobs every day. Operating System: Windows, Linux, OS X

Developed by LinkedIn and open sourced in June 2015, Pinto is a distributed OLAP datastore that enables real-time scalable analytics. Key features include column orientation, pluggable indexing technology, data ingestion from Kafka and Hadoop and support for a SQL-like language. Operating System: Windows, Linux, OS X

Pivotal, a spinoff of EMC and VMware, announced earlier this year that they would be open sourcing key pieces of their big data suite, including GemFire, Greenplum Database and HAWQ. Gemfire helps organizations build data-driven applications, Greenplum is an analytics-enabled database, and HAWQ is a SQL analytics engine for Hadoop. Operating System: Windows, Linux, OS X

This Netflix project provides an easy-to-understand graphical representation of Hadoop Pig jobs. It updates as the job executes so that administrators and developers no longer need to sift through log data. Operating System: Windows, Linux, OS X

Originally developed by LinkedIn, this Apache top-level project does distributed stream processing. It relies on the Kafka messaging system as well as Hadoop YARN distributed processing technology. Operating System: Windows, Linux, OS X

Machine Learning

LinkedIn first released this project earlier this month. According to the company it is "a new open source toolkit designed to enable creative and agile feature engineering for most machine learning tasks such as statistical modeling (classification, clustering, and regression) and rule-based decision engines." Operating System: Linux

Created by financial services heavyweight Capital One, Hygieia (pronounced "hi-gee-ya") is a DevOps dashboard that allows users to see the status of their entire delivery pipeline at a glance. The code, screenshots and a video explaining the tool are available on Capital One's Git Hub pages. Operating System: OS Independent