Continuuity decided to build Weave and be part of the journey to take Apache YARN to the next level of usability and functionality. Continuuity has been using Weave extensively to support their products and seen the benefit and power of Apache YARN and Weave combined. Continuuity decided to share Weave under the Apache 2.0 license in an effort to collaborate with members of the community, broaden the set of applications and patterns that Weave supports, and further the overall adoption of Apache YARN.

Weave is NOT a replacement for Apache YARN. It is instead a value-added framework that operates on top of Apache YARN.

What is Weave: Weave is a simple set of libraries that allows you to easily manage distributed applications through an abstraction layer built on Apache YARN. Weave allows you to use YARN’s distributed capabilities with a programming model that is similar to running threads.

Features of Weave:
– Simple API for specifying, running and managing application lifecycle
– An easy way to communicate with an application or parts of an application
– A generic Application Master to better support simple applications
– Simplified archive management and local file transport
– Improved control over application logs, metrics and errors
– Discovery service
– And many more…

Once you figure out above, you would need to follow below steps to copy above SSH authorized_keys to same location in your new user home folder. In example below the new user account is amazonuser while main account user is ubuntu:

When setting Hadoop cluster in Amazon EC2 you would need to configure proper security settings (firewall) so you can access Hadoop cluster directly. Following are the settings for Cloudera CDH4 Hadoop distribution on EC2:

Port 22 for SSH, Port 7180/82 for CDH Manager, 7432 for PSQL and 8888 for Hue and finally Port 50000-50100 for Hadoop JT and HDFS.

While creating a Linux instance at Amazon EC2 you can choose to “Create and Download SSH Key” option which will let you download a PEM file (SSH Secure Key) on your local machine and you can use this SSH secure key while connecting your Linux instance at Amazon.

Once your Linux instance is ready and running at Amazon you can get instance URL from Amazon portal shown as below:

You can use iTerm/Terminal (Mac) or Putty/BitVise SSH client (Windows) applications to connect your Linux instance as below:

1. Change your PEM file permissions

$chmod -R 700 Downloads/your_instance_ssh_key.pem

2. Now you can use the ssh command to connect to your Linux instance by using -i parameter to use key: