Application Development on Apache YARN Simplifies with Release of Weave

Developers love the latest and greatest development tooling. And next up it’s Kafka, a real-time framework for managing data streams or YARN (or Yet Another Resource Manager) aka MapReduce 2.0, the next generation processing framework for Apache Hadoop.

Continuuity, the provider of scale-out application server for Apache Hadoop, announced the release of Weave, a new open source project for Java developers to build, deploy, scale and manage scalable, distributed applications on YARN.

Weave is a simple set of libraries for easy management of distributed applications on Apache YARN. The tool allows you to build a much wider class of applications on top of YARN. The continued popularity of the JVM (Java Virtual Machine) has breathed new life into languages. Weave is similar to a local JVM as it makes developing Java-based YARN applications even easier.

“We developed Apache Hadoop YARN to support a wide set of use cases in a generic manner, and it is very important to make it easy for developers to exploit the features and capabilities of YARN. We are excited to see the new types of applications that can be built on YARN using Weave,” said Arun Murthy, co-founder of Hortonworks. “We are thrilled that Continuuity has decided to help make YARN even easier to use, and that they are sharing their efforts with the community.”

YARN is used to run and manage BigFlow applications and is extremely powerful, but it is difficult to get test and debug, and complex to build new kinds of non-MapReduce applications and frameworks on it. Weave, under the Apache 2.0 license, extends the benefit and power of Apache YARN and use YARN’s distributed capabilities with a programming model that is similar to running threads.

“Continuuity’s mission is to make building Big Data applications easier for developers. With the release of Weave, we’re supporting an increasingly diverse set of real-time application patterns to spur innovation in the Hadoop ecosystem,” said Jonathan Gray, CTO and co-founder of Continuuity. “Our goal is to continue collaborating with the community and improving Weave and YARN.”

Weave features simple API, simplified archive management, improved control over application logs, metrics and errors, better communicate with an application and easy communication between containers and the Application Master.