Hortonworks Manifesto

We’re glad to have finally launched Hortonworks after months of planning and speculation. I thought I’d use the opportunity of my first Hortonworks blog to lay out who we are and what we’re all about.

Our History

Hortonworks was formed by Yahoo! and Benchmark Capital in June 2011 in order to accelerate the development and adoption of Apache Hadoop. We believe that Apache Hadoop will become the de facto platform for storing, managing and analyzing “big data,” namely the exploding volume of data being generated daily by organizations around the globe.

As one of the creators, the primary contributor to, and one of the leading users of Apache Hadoop, Yahoo! has extensive experience in realizing exceptional business value from the Hadoop platform. In fact, Hadoop is now behind every click at Yahoo!, running on 42,000 servers and delivering personalized content and experiences to nearly 700 million consumers worldwide. Apache Hadoop helps drive Yahoo!’s powerful advertising platform and enables Yahoo! to provide enhanced anti-spam capabilities in Yahoo! Mail, among many other uses.

Yahoo!’s motivation in forming Hortonworks with Benchmark Capital was to create an independent entity that could focus development efforts on improving the Apache Hadoop platform, working in close concert with the rest of the Apache community. At Hortonworks, we are focused on removing current technology and knowledge gaps to help Apache Hadoop gain mass appeal and adoption among enterprises and technology vendors alike. We want to make Apache Hadoop more robust, make it easier for companies to install, manage and use it and make it easier for technology vendors to integrate and extend it. Quite simply, we want to help other organizations derive similar value to what Yahoo! has experienced, regardless of whether they are running Apache Hadoop on a single cluster or across 42,000 servers.

Our Mission

Our mission is simple: To revolutionize and commoditize the storage and processing of big data via open source.

In order to accomplish this mission, we are focused on accelerating the development and adoption of Apache Hadoop. By making it easier to consume for enterprises and technology vendors, we believe we can overcome any technology and knowledge gaps that could keep Apache Hadoop from reaching its potential. We believe that more than half of the world’s data will be processed in Apache Hadoop within five years.

What Motivates Us

Quite simply, these three things:

1. Architecting the Future of Big Data – We believe Apache Hadoop is the ideal platform for storing, managing and analyzing big data. We have already invested a lot of blood, sweat and tears in Apache Hadoop and we take tremendous pride in helping to define its future.

2. Solving Real-World Business Challenges – While at Yahoo!, the core Hortonworks development team gained extensive experience applying Apache Hadoop technology to deliver real business value. We want to share this knowledge and experience with other organizations, large and small.

3. Open Source – We believe that open source licensing via the Apache Software Foundation is the best way to solve the big data challenge. Open source allows the largest possible user community to try Apache Hadoop for free, to validate that our code does what we say it does and to know that our work will have longevity beyond that of any one company or contributor.

Our Team

We have an exceptional amount of development and open source talent and a passion for creating difference-making technology. The core team consists of many of the key architects and developers from the Yahoo! Hadoop engineering team, the same team that pioneered Apache Hadoop and delivered every stable release from 0.1 to the most current release. We also have the largest concentration of Apache Hadoop committers who contribute a significant amount of source code to core Apache Hadoop projects including HDFS, MapReduce, Pig and others. Among the most well known contributors are our other founders: Alan Gates, Arun Murthy, Devaraj Das, Mahadev Konar, Owen O’Malley, Sanjay Radia and Suresh Srinivas.

Hortonworks also has significant talent on the business side, including Rob Bearden, the former COO of both SpringSource and JBoss, two of the most successful commercial open source companies in history. We are also backed by Benchmark Capital, the preeminent technology VC firm that has also invested in and helped grow technology companies ranging from Red Hat, MySQL and SpringSource to Twitter and eBay.

Our Core Values

We believe strongly in our core set of shared values. These values shape who we are and how we make decisions on a daily basis. At Hortonworks, we are:

* Passionate about our mission, customers and community. We are excited about the impact that our work can have on the world and each of us is committed to delivering great work that makes us all proud.

* Delivery-focused. We do what we promise. No excuses.

* Thought leaders, designers and builders. We drive innovation in our community and in all of our activities. We hire smart, innovative people who have a history of delivering great work.

* Clear and direct. We tell it straight and stick to our word. We don’t tolerate politics or hidden agendas.

* Team players. We don’t tolerate prima donnas. We strive to remain humble and celebrate each other’s successes.

* Transparent (everyone has the maps). We use open processes and planning to engage our community, partners and customers. We share information widely and openly within the company. We expect team members to determine the right thing to do and go do it.

* Relentlessly positive. We are polite and positive with each other, and with our customers, partners and community. We strive to improve our reputation with every public interaction.

* Always looking for good ideas. Good ideas can come from anywhere and anyone. We strive to be good listeners.

* Willing to recognize and fix our mistakes. Because we take risks, we will make some mistakes. We promise to learn from each mistake and not make the same mistakes twice.

* In love with simplicity (KISS). Genius lies in solving hard problems very simply. We pick our battles. We don’t tilt at windmills.

* Aware that there is more to life than work. We believe work should be fun and we won’t take ourselves too seriously.

Our Commitment to Open Source

We are committed and passionate about open source. We believe that the reference implementation of Hadoop should come from Apache, and we are committed to ensuring that remains a reality. We believe anyone should be able to go to Apache and download a functioning, tested and stable version of Hadoop that can be deployed in enterprise environments to solve real-world problems. We will do our development in public, releasing early and often via Apache.

We will continue to be a driving force behind Apache Hadoop and major contributors to core projects including Hadoop Core, MapReduce, Hadoop Distributed File System (HDFS)(tm), ZooKeeper(tm) and Pig(tm). We are committed to working with the community to build and productize the next generation of Apache Hadoop.

While we may build applications or tools in the future that are only available to paying customers, our commitment is that all core code remain Open Source.

Our Technology

There is really no such thing as “our technology.” Together with the Apache Hadoop community, we design and develop software that is contributed back to Apache in its entirety.

Our development focus covers the following areas:

* Make Apache Hadoop easier to install, manage and use – we have a mantra that anyone should be able to easily deploy the Hadoop projects directly from Apache. We are focused on making this a reality. This includes regular sustaining releases, availability of compiled code (binary formats such as RPMs, .debs, etc.), improved QA and testing at scale.

* Make Apache Hadoop more robust – Apache Hadoop is increasingly being used in mission critical environments, which has created even more demand to improve the enterprise robustness of the platform. We are driving the design and development of new generations of Apache Hadoop that will address important areas such as performance, availability, scalability and security. We will also work with the community to add new capabilities to open source including administration and monitoring to name a few.

* Make Apache Hadoop easier to integrate and extend – we will work closely with our strategic partners and the larger community to identify opportunities to simplify how ISVs and other technology vendors add value to overall ecosystem. This will include open APIs that will help organizations extend and experiment with Apache Hadoop.

Our Business Strategy

Our goal is to enable IT teams to create new and compelling business value at their organizations. Hortonworks will offer technology and services that make it easier, more manageable and cost-effective to store, manage, process and analyze the ever-increasing amount of data flowing into and throughout organizations.

We will share more information about our specific offerings in the near future including support subscriptions, training and certification for developers, architects and administrators at enterprises, systems integrators and technology vendors such as ISVs, OEMs and service providers.

Our Long-Term Objectives

We believe strongly that Apache Hadoop will be a critical and foundational component within any data-driven organization’s IT architecture. Just like enterprise data warehouses, data marts and relational databases are critical for structured and transactional data, Apache Hadoop will become the platform by which these same organizations store, manage, process and analyze the faster growing area of semi-structured and un-structured data. We believe that more than half of the world’s data will be processed in Hadoop within five years.

We are dedicated to building and delivering current and future generations of Apache Hadoop, working closely with the community and donating all core source code back to Apache. To augment the core Apache Hadoop technology, we will deliver an ever-increasing array of services aimed at improving the Hadoop experience and supporting the growing needs of enterprises, system integrators and technology vendors including ISVs, OEMs and service providers.