Open Source Communities: The Developer Kingdom

Since our founding in 2011, Hortonworks has had a fundamental belief: the only way to deliver infrastructure platform technology is completely in open source. Moreover, we believe that collaborative open source software development under the governance model of an entity like the Apache Software Foundation (ASF) is the best way to accelerate innovation that targets enterprise end users since it brings the largest number of developers together in a way that enables innovation to happen far faster than any single vendor could achieve and in a way that is free of friction for the enterprise.

In the past, open source development was thought of as a bunch of hackers living in basements or working out of garages. Fast-forward to 2015 and, as Stephen O’Grady puts it, developers “have become the most important constituency in business seemingly overnight.” They are some of the highest paid and most valued assets in any modern organization. Developers are the new kingmakers.

No matter if they are individual developers or rockstars in a mega corporation, many of these key talents have found each other in a common and very open community. They are contributing and building out key platform technologies such as Apache Hadoop.

But why? Why contribute? Why give away such important intellectual property?
With platform technologies, the developer just wants it to work and the mega corporation just wants to pursue adjacent opportunities built on it. Every company is becoming a software company and the platform on which they build needs to be stable, reliable and complete. So, in the end, what is good for the platform is GREAT for all.

None of this can happen without the ASF

The ASF is critical in providing the right environment where meritocracy rules and platform technologies can be advanced. It provides valuable stewardship and guide-rails for projects interested in attracting the broadest community of involvement as possible.

The ASF recently published the Apache Project Maturity Model document that provides a GREAT overview of “how Apache projects operate, in a concise and high-level way”. It is an important piece of work in this new world of software development, and we encourage open source projects without a clearly defined governance model like the ASF’s to read through and use some of these items in support of your own project’s goals.

The Hortonworks development model complements what the ASF provides by defining how the dozens of ASF projects included within the Hortonworks Data Platform (HDP) should work together in concert as part of an enterprise-viable data platform. Understanding and satisfying enterprise requirements that span the ASF projects and/or making tradeoffs about which projects are better suited for specific use cases are ways we enable value above and beyond what the ASF focuses on.

Our development model enables us to get the latest stable innovation into our end users’ hands quickly. It’s been designed to enable a virtuous cycle of open source innovation that flows naturally from upstream ASF projects into downstream enterprise grade HDP releases. This minimizes and/or eliminates drift from the corresponding stable ASF project releases. We also work diligently to ensure bug fixes included in HDP 1.x and 2.x maintenance releases are managed in a way that avoid regressions in future releases.

Our certification process is underpinned by a test suite unique to Hortonworks that’s comprised of 10’s of thousands of functional, system, and integration tests that are run on some of the world’s largest clusters within our joint engineering partners’ environments across a wide range of Linux and Windows systems. By doing all feature development and bug fix work in the upstream ASF projects, our downstream HDP process can cleanly harmonize a set of stable ASF project releases into a rigorously tested and certified platform suitable for the enterprise.

Focusing on enabling an equitable balance of power, not lock-in

End users have learned that open source can not only meet their requirements, but can also enable them to get out from under the yoke of any single vendor’s agenda. Moreover, active participation in open source is not just the domain of web-scale end users like Yahoo!, Google, Facebook, Netflix, and LinkedIn.

Mainstream enterprises such as Aetna, Merck, and Target are participating in open source initiatives that address such needs as data governance since they are vital to their businesses and industries. While our customers are normally downstream consumers of HDP and our subscription services, efforts like the Data Governance Initiative, are enabling customers to move upstream into the ASF process to help prioritize features and develop/test the ASF project code so the released technologies are highly aligned with their requirements.

Since we make HDP available free of charge and derive the majority of our revenue from annual support subscriptions, we realize that “every year is an election year” for our customers. We feel our open source development model yields unmatched efficiency and customer satisfaction since it provides a direct way for customers and partners to participate, drive value, and capture adjacent opportunities.

And we feel our business model establishes an equitable balance of power that de-risks investments and focuses on mutual success.

A Benevolent Benefactor

We at Hortonworks have been part of the open source Apache Hadoop movement from its very beginnings. Our team members have helped incubate many new open source technologies and are well versed in how communities work and how organizations can benefit from them. We endeavor to help guide and extend the Hadoop ecosystem as a benevolent benefactor, ensuring all those in the data center can benefit from it.

Another key principle we were founded on was to enable the ecosystem to adopt and extend Hadoop. We do so in the way we help architect solutions. We do so in the way we partner with the technology leaders. We do so in the way we get involved and represent our customer in the community. And it’s all fueled by the rock star developers we employ and the communities in which they work.

If you’d like to see the community in action for yourself, then please join us at Hadoop Summit Brussels and San Jose where you can engage with end users and the broader Hadoop community to help shape the future of Apache Hadoop!

Tags:

Your email address will not be published. Required fields are marked *

Comment

If you have specific technical questions, please post them in the Forums

Name*

Email*

Related Posts

BLOG

6.9.17

TMW: From Adoption to Profitability

The San Jose DataWorks Summit is this Tuesday! Have you registered? A few weeks ago, we announced Tim Leonard of TMW Systems would be presenting a Tuesday keynote, Commoditizing Your Data to Sell – A Transportation Example. Lessons from this keynote are vital to any small business looking to compete with major players in an industry.…

Announcing 2017 European Hortonworks Data Heroes...

Not only will DataWorks Summit/Hadoop Summit Munich be my first in Europe since joining Hortonworks but it is also the global launch of our Data Heroes Awards. On the evening before each of the three conferences begin, we will both recognise our nine finalists and announce the winners from each category! With the awards, Hortonworks…

Hortonworks 2016 Year in Review

As we kick off the new year I wanted to thank our customers, partners, Apache community members, and of course the amazing Hortonworks team, for an amazing 2016. Let’s take a step back and look at some of the Hortonworks highlights from last year... IN THE ECOSYSTEM there was tremendous acceleration. At the beginning of…

The Power of your Data Achieved:...

It’s no secret that there is a data explosion. A recent IDC analyst report from April 2014 indicated the volume of data, known as the digital universe, is doubling in size every two years. And by 2020, there will be as many digital bits as there are stars in the universe. There are many reasons…

Jumpstart Your Digital Transformation with Hadoop...

Guest author: Jeff Kelly, Data Strategist, Pivotal The phrase “digital transformation” gets bandied about a lot these days, but what exactly does it mean? When you strip away the hyperbole, I believe digital transformation is the process by which enterprises evolve from using traditional information technology to merely support existing business models to adopting modern…

What’s the best cloud architecture—and how...

People often think about cloud architecture in simplistic terms: you’re either public, private, or hybrid. (In fact, there’s even confusion about the meaning of the term “hybrid” itself—this video helps clear it up: https://www.youtube.com/watch?v=HPKI-U_ef5w In the real world, of course, virtually every implementation is hybrid—no company puts 100% of its IT environment into one single…

The 100% open source and community driven innovation of Apache Hive 2.0 and LLAP (Long Last and Process) truly brings agile analytics to the next level. It enables customers to perform sub-second interactive queries without the need for additional SQL-based analytical tools, enabling rapid analytical iterations and providing significant time-to-value. TRY HIVE LLAP TODAY Read about…

If You Think Cloud, Think Connected...

Cloud Computing is one of the big three trends impacting IT architectures today. What some may not realize is that an underlying connected data architecture is not only essential for cloud, but sits at the confluence of all three trends. Here's why. The first big trend is IoT. According to BI Intelligence, we can now…

Insights Aggregation and Predictive Analytics within...

How Hortonworks can help hotel industry capture value through Insights Aggregation and Predictive Analytics Big Data has transformed every industry including the hospitality vertical. Through customer analytics, targeted segmentation, and campaigning, hotels would like to focus on delivering personalized promotions, cross and up-selling travel services. Our objective is to address these challenges through an open-source…