What is Amazon EMR?

With Amazon Elastic MapReduce (Amazon EMR) you can analyze and process vast amounts of data. It does this by
distributing the computational work across a cluster of virtual servers running in the
Amazon cloud. The cluster is managed using an open-source framework called Hadoop.

Hadoop uses a distributed processing architecture called MapReduce in which a task is
mapped to a set of servers for processing. The results of the computation performed by those
servers is then reduced down to a single output set. One node, designated as the master
node, controls the distribution of tasks. The following diagram shows a Hadoop cluster with
the master node directing a group of slave nodes which process the data.

Amazon EMR has made enhancements to Hadoop and other open-source applications to work
seamlessly with AWS. For example, Hadoop clusters running on Amazon EMR use EC2 instances as
virtual Linux servers for the master and slave nodes, Amazon S3 for bulk storage of input and
output data, and CloudWatch to monitor cluster performance and raise alarms. You can also move
data into and out of DynamoDB using Amazon EMR and Hive. All of this is orchestrated by Amazon EMR
control software that launches and manages the Hadoop cluster. This process is called an
Amazon EMR cluster.

The following diagram illustrates how Amazon EMR interacts with other AWS services.

Open-source projects that run on top of the Hadoop architecture can also be run on Amazon EMR.
The most popular applications, such as Hive, Pig, HBase, DistCp, and Ganglia, are already
integrated with Amazon EMR.

By running Hadoop on Amazon EMR you get the benefits of the cloud:

The ability to provision clusters of virtual servers within minutes.

You can scale the number of virtual servers in your cluster to manage your computation needs, and only pay for what you use.