Announcing the Firecracker Open
Source Technology: Secure and Fast microVM for Serverless Computing

By AWS' Arun Gupta and Linda Lian

November 28, 2018

New Challenges for
Virtualization

Today, our customers
can use serverless computing to build applications without worrying
about provisioning or managing infrastructure. Developers can
package their code as serverless containers with AWS Fargate or
serverless functions with AWS Lambda. Our customers tell us they
love the low operational overhead of serverless, and we believe it
will continue to play a pivotal role in the future of computing.

As our customers
increasingly adopted serverless, we realized that existing
virtualization technologies were not developed to optimize for the
event-driven, sometimes short-lived nature of these kinds of
workloads. We saw a need to build virtualization technology
specifically designed for serverless computing. We needed something
that could give us the hardware virtualization-based security
boundaries of virtual machines, while maintaining the smaller
package size and agility of containers and functions.

Firecracker
Technology

Meet Firecracker,
an open source virtual machine monitor (VMM) that uses the Linux
Kernel-based Virtual Machine (KVM). Firecracker allows you to create
micro Virtual Machines or microVMs. Firecracker is minimalist by
design – it includes only what you need to run secure and
lightweight VMs. At every step of the design process, we optimized
Firecracker for security, speed, and efficiency. For example, we can
only boot relatively recent Linux kernels, and only when they are
compiled with a specific set of configuration options (there are
1000+ kernel compile config options). Also, there is no support for
graphics or accelerators of any kind, no support for hardware
passthrough, and no support for (most) legacy devices.

Firecracker boots a
minimal kernel config without relying on an emulated bios and
without a complete device model. The only devices are virtio net and
virtio block, as well as a one-button keyboard (the reset pin helps
when there’s no power management device). This minimal device model
not only enables faster startup times (< 125 ms on an i3.metal with
the default microVM size), but also reduces the attack surface, for
increased security. Read more details about Firecracker’s promise to
enable minimal-overhead execution
of container and serverless workloads.

In the fall of 2017,
we decided to write Firecracker in Rust,
a modern programming language that guarantees thread and memory
safety and prevents buffer overflows and many other types of memory
safety errors that can lead to security vulnerabilities. Read more
details about the features and architecture of the Firecracker VMM
at Firecracker Design.

Firecracker microVMs
improve efficiency and utilization with a low memory overhead of < 5
MiB per microVMs. This means that you can pack thousands of microVMs
onto a single machine. You can use an in-process rate limiter to
control, with fine granularity, how network and storage resources
are shared, even across thousands of microVMs. All hardware compute
resources can be safely oversubscribed, to maximize the number of
workloads that can run on a host.

We developed
Firecracker with the following guiding tenets
(unless you know better ones) for the open source project:

Built-In
Security: We provide compute security barriers that enable
multitenant workloads, and cannot be mistakenly disabled by
customers. Customer workloads are simultaneously considered
sacred (shall not be touched) and malicious (shall be defended
against).

Light-Weight
Virtualization: We focus on transient or stateless workloads
over long-running or persistent workloads. Firecracker’s
hardware resources overhead is known and guaranteed.

Minimalist in
Features: If it’s not clearly required for our mission, we
won’t build it. We maintain a single implementation per
capability.

Compute
Oversubscription: All of the hardware compute resources
exposed by Firecracker to guests can be securely oversubscribed.

We open sourced this
foundational technology because we believe that our mission to build
the next generation of virtualization for serverless computing has
just begun.

Firecracker Usage

AWS Lambda uses
Firecracker as the foundation for provisioning and running sandboxes
upon which we execute customer code. Because Firecracker provides a
secure microVM which can be rapidly provisioned with a minimal
footprint, it enables performance without sacrificing security. This
lets us drive high utilization on physical hardware, as we can now
optimize how we distribute and run workloads for Lambda, mixing
workloads based on factors like active/idle periods, and memory
utilization.

Previously, Fargate
Tasks consisted of one or more Docker containers running inside a
dedicated EC2 VM to ensure isolation across Tasks. These Tasks now
execute on Firecracker microVMs, which allows us to provision the
Fargate runtime layer faster and more efficiently on EC2 bare metal
instances, and improve density without compromising kernel-level
isolation of Tasks. Over time, this will allow us to continue to
innovate at the runtime layer, giving our customers even better
performance while maintaining our high security bar, and lowering
the overall cost of running serverless container architectures.

Firecracker runs on
Intel processors today, with support for AMD and ARM coming in 2019.

You can run
Firecracker on AWS .metal instances, as well as on any
other bare-metal server, including on-premises environments and
developer laptops.

Firecracker will also
enable popular container runtimes such as containerd to manage
containers as microVMs. This allows Docker and container
orchestration frameworks such as Kubernetes to use Firecracker. We
have built a prototype
that enables containerd to manage containers as Firecracker microVMs
and would like to with with community to take it further.

Getting Started with
Firecracker

Getting Started with
Firecracker
provides detailed instructions on how to download the Firecracker
binary, start Firecracker with different options, build from the
source, and run integration tests. You can run Firecracker in
production using the Firecracker Jailer.

Let’s take a look at
how to get started with using Firecracker on AWS Cloud (these steps
can be used on any bare metal machine):

Create an
i3.metal instance using Ubuntu 18.04.1.

Firecracker is built
on top of KVM and needs read/write access to /dev/kvm.
Log in to the host in one terminal and set up that access:

This starts a VMM
process and waits for the microVM configuration. By default, one
vCPU and 128 MiB memory are assigned to each microVM. Now this
microVM needs to be configured with an uncompressed Linux kernel
binary and an ext4 file system image to be used as root filesystem.

Log in as root
with password root to see the terminal of the guest
machine:

Bash

localhost login: root
Password:
Welcome to Alpine!
The Alpine Wiki contains a large amount of how-to guides and general information

about administrating Alpine systems.
See <http://wiki.alpinelinux.org>.
You can setup the system with the command: setup-alpine
You may change this message by editing /etc/motd.
login[979]: root login on 'ttyS0'
localhost:~#

Terminate the microVM
using the reboot command. Firecracker currently does
not implement guest power management, as a tradeoff for efficiency.
Instead, the reboot command issues a keyboard reset action which is
then used as a shutdown switch.

Once the basic
microVM is created, you can add network interfaces, add more drives,
and continue to configure the microVM.

Multiple microVMs may
be configured with a single shared root file system, and each
microVM can then be assigned its own read/write share.

Firecracker and Open
Source

It
is our mission to innovate on behalf of and for our customers, and
we will continue to invest deeply in serverless computing at all
three critical layers of the stack: the application, virtualization,
and hardware layers. We want to offer our customers their choice of
compute, whether instances or serverless, with no compromises on
security, scalability, or performance. Firecracker is a fundamental
building block for providing that experience.

Investing deeply in
foundational technologies is one of the key ways that we at AWS
approach innovation – not for tomorrow, but for the next decade and
beyond. Sharing this technology with the community goes hand-in-hand
with this innovation. Firecracker is licensed under Apache 2.0.
Please visit the Firecracker GitHub repo
to learn more and contribute to
Firecracker.

By open sourcing
Firecracker, we not only invite you to a deeper examination of the
foundational technologies that we are building to underpin the
future of serverless computing, but we also hope that you will join
us in strengthening and improving Firecracker. See the Firecracker issues list
and the Firecracker roadmap
for more information.