A Comparison of Serverless Function (FaaS) Providers

Brecht DeRooms Feb 06, 2020

New Function-as-a-service (FaaS) providers are rising and gaining adoption. How do these new providers differentiate
themselves from the existing big players? We will try to answer these questions by exploring the capabilities of both
the more popular FaaS offerings and the new players. We will look at where their focus lies, the limitations, what
programming languages they support, and their pricing models. The goal is to build a reference for those looking to
choose between providers. Since this should be a collaborative effort, feel free to contact us if you would like to
see something added.

Before we dive in, let’s briefly discuss the advantages of FaaS over other solutions. Why would you choose to go
serverless?

How does FaaS compare to PaaS and CaaS

Functions as a Service (FaaS) vs. Platform as a service (PaaS)

At the highest level, the choice between PaaS and FaaS is a choice of control versus ease of use, and a choice
between architectures (monolith versus microservices). A platform such as Heroku is the first choice for many proof of
concepts because getting started is easy and it has powerful tooling to make a developer’s life easier. Typically,
such applications are monoliths that scale by adding nodes with another instance of the whole application (e.g., in
Heroku terms: add more dynos). An application can go a long way on such a platform without running into performance
problems. Still, if one part in your code becomes a bottleneck, this monolith approach makes it harder to scale your
application. Once the load on your application increases, PaaS providers typically become expensive (1).

FaaS, on the other hand, requires you to break up applications into functions, resulting in a microservices
architecture. In a FaaS-based application, scaling happens at the function level, and new functions are spawned as
they are needed. This lower granularity avoids bottlenecks, but also has a downside since many small components
(functions) require orchestration to make them cooperate. Orchestration is the extra configuration work that is
necessary to define how functions discover each other or talk to each other.

Advantages

Disadvantages

Platforms (PaaS)

very easy to get started

less control, especially over scaling

in general, more expensive

vendor lock-in

your process can go down

Functions (FaaS)

more control, less flexibility

a function that crashes won’t bring down your app

the orchestration of many functions results in more configuration

Containers (CaaS) vs Functions (FaaS)

Although containers and functions solve a very different problem, they are often compared since the type of
applications that are built with them can be very similar. Both provide ways to easily deploy a piece of code. In the
container case, one typically still has to decide how many instances need to run while functions are 100%
auto-scaling. There will always be one container running for a specific service, waiting until it receives a task,
while a function only runs on request. Containers start up fairly quickly, but are not fast enough to catch up with
quick bursts in traffic. In contrast, functions just scale along with the burst.

In general, containers require much more configuration, especially because scaling rules must be tweaked in order not
to over- or underprovision. In FaaS, over- or underprovisioning becomes the concern of the provider while clients are
provided with a pay-as-you-go model. The pay-as-you-go model is quickly becoming the definition of "serverless" since
it abstracts away the last indication of servers from the developer. Of course, the servers that execute your
functions are still there and the provider needs to make sure that there are enough of them when a client launches new
functions. In order to make this manageable, there are typically memory, payload, and execution time limits in place.
These limits make it easier for FaaS providers to estimate the load at a given time and are the prices we pay to be
able to run code without caring about servers.

Advantages

Disadvantages

Containers (CaaS)

portability (can run everywhere)

flexibility in environment and language

no vendor lock-in

no cold starts, if you keep the containers running

does not suffer from the typical FaaS memory/payload/execution limits

pay for what you don’t use.

scaling is slower

scaling configuration required

need to install your environment

your process can still go down since containers are meant to keep running

Functions (FaaS)

granular scaling

no need to install software

functions don't go down, a new function will run anyway when one crashes

no scaling configuration

faster scaling

pay-as-you-go

cold starts

high granularity, the orchestration of many functions can result in more configuration then needed

memory, payload, and execution limits

FaaS providers compared

Focus: ease-of-use vs configurability vs edge computing. Of
course, people choose a cloud ecosystem for different reasons (e.g., free Azure credits, deployment via
CloudFormation, or a specific feature like Google Dataflow) and the logical choice is usually the provider you already
use. If the ecosystem is not the deciding factor, the focus of the FaaS provider is often the reason why a company
chooses a particular provider.

Generality and configurability:

The biggest providers, such as Azure, Google, and AWS, focus on
configurability. Their function offerings are often a vital part of communication between other services of their
ecosystem. The trigger mechanism is therefore separated from the function allowing the function to be activated by
many things such as database triggers, queue triggers, custom events based on logs, scheduled events, or load
balancers. If all you need is a REST API based on functions, then this might seem cumbersome. For example, when you
write an API based on AWS Lambda functions, you need to write the function, deploy it, set up security rules to be
able to execute them, and configure how they are triggered (e.g., via a load balancer). The number of configuration
possibilities can make the documentation seem daunting.

Easy-of-use and developer experience:

Other providers such as Netlify and Vercel (formerly ZEIT) respond to
that by focusing on a more narrow use case and ease-of-use. In this case, one command is often enough to transform the
function into a REST or GraphQL API that is ready to be consumed. The starting barrier and learning curve is greatly
reduced because the functions serve a more specific purpose (APIs). Besides that, they provide an impressive toolchain
to easily debug, deploy, and version your functions. For example, both support file system routing which allows you to
simply drop a function in a folder so that it becomes accessible as an API with the same path, which also works
locally. Since the functions live on the same origin as your frontend code, you do not have to deal with CORs issues.

Their function offering also integrates perfectly with the rest of their offerings, which are aimed at facilitating
JAMstack applications and typical Single Page Applications (SPAs). JAMstack sites are relatively static sites that
make heavy use of serverless offerings to populate the smaller dynamic parts.

Vercel Functions: Deploy with the Now ecosystem and you have a scalable
API under the /api endpoint that can also be simulated locally. When using Functions inside a Next.js app, you receive the benefits of Webpack and Babel and do not have to deal
with response payload limits since VercelFunctions don’t have these limits.

Netlify Functions: Deploy with the Netlify CI by just pushing your code to Github. By integrating with their
Identify product (similar to Auth0), Functions automatically receive user information. Netlify also provides
one-click add-ons such as databases from which the environment variables will be automatically injected.

You can say that Netlify and Vercel compare to AWS/Azure Functions/Google Functions as a PaaS platform like Heroku
compares to AWS: they abstract away the complexities of functions to provide a very smooth developer experience with
less setup and overhead. Under the hood, they both rely on AWS to run their functions. Their free tier does not only
include function invocations; it also comes with everything you need to deploy a website such as hosting, CI builds,
and CDN bandwidth.

Edge computing

If low latency is necessary and data always needs to be real-time, then Cloudflare Workers, EdgeEngine, Lambda@Edge, or Fly.io are probably the
best choices. Their focus is essentially edge computing, or in other words, bringing the functions as close as
possible to the end-users to reduce latency to an absolute minimum. At the time of writing (November, 2019),
Cloudflare Workers have 194 points of presence, EdgeEngine sports 45+ locations, and Fly.io is at 16 locations.

Most of them run functions straight on the V8 engine instead of NodeJS, which allows for lower latency than their
competitors. Due to that, they initially only supported JavaScript but all have recently added support for WebAssembly which should (theoretically) allow most languages. Fly.io recently dropped the V8 runtime to allow for more intensive workloads and worked around cold-starts by keeping processes running. EdgeEngine and Cloudflare
Workers are probably not meant for very CPU intensive tasks since the runtime limitations are expressed in CPU time. A
function can exist for a long time, as long as it does not actively use the CPU longer than the runtime limitation.
For example, an idle function that is waiting for the result of a network call is not spending any CPU time. This makes these providers a great choice to build a 'middleware' backend that spends most of its time forwarding and waiting on requests.

Fly.io differentiates by focusing on a powerful API and an open-source runtime which allows you to build and test your apps in a local environment. In a recent update, they moved away from the V8 engine in favour of docker which removed typical serverless limits and explains why their pricing is now longer based on the duration of a call. This turns Fly.io in a service to run docker images on servers in different cities. This service comes with a global router to connect users to the nearest available instance and the automatic addition of VMs on popular locations as your application's traffic increases. This makes Fly.io a great service to run heavier tasks close to the edge such as a full-fledged rails application.

It is noteworthy that Vercel provides a unique Edge
Caching system which approximates the serverless edge experience. This system caches data from your serverless
functions at configurable intervals, which gives users fast data access, although the data is not real-time.

Do it yourself

Finally, there is a whole new set of offerings that are starting to move away from the serverless aspect of FaaS.
Many have bumped into the limits (payload, memory) of functions and then tried to get around those limits. The typical
solution was to start running Docker containers. Frameworks like OpenFaaS, the FN Project, Fission, OpenWhisk, and
Kubeless are aiming to provide a framework that allows you to deploy your own FaaS solution. They aim to deliver a
similar experience to a true FaaS provider, but it’s important to note that this FaaS is all but serverless since you
are again responsible for managing clusters and scaling. One can argue that deploying Kubeless on a managed Kubernetes
service comes very close and it would not be surprising if these open source frameworks motivate other companies to
start their own FaaS offerings. In fact, some of these already serve as the basis for a commercial service provider.
OpenWhisk powers IBM's FaaS offering while a fork of the Fn project is the technology behind Oracle Functions. Fly.io
also deserves a space in this category since they rolled out an open source runtime. The future will definitely be
interesting; we expect to see FaaS offerings for each niche.

Comparing the focus of FaaS providers

Do it yourself

Open-

FaaS

Fn project

Fission

Open-

Whisk

Kubeless

Fly.io*

Configurability

AWS

Lambda

Google

Functions

Azure

Functions

IBM

Cloud Functions

Alibaba

Functions

Oracle

Functions

Ease-of-use

Vercel Functions

Netlify Functions

Fly.io*

Edge computing

Cloudflare Workers

Edge

Engine

Fly.io

Lambda

@Edge

Vercel Functions + Edge

Cache*

A table categorizing all FaaS providers according to their main focus

Providers starred with * have noteworthy features in that domain although it’s not their main focus

Limits

One of the main pitfalls of using Functions as a Service is the limits. Some developers are not aware of those when
they get started, or they underestimate how easily an application can bump into these limits over time. The number of
workarounds that can be found online where results are stored temporarily on another location such as AWS S3 shows how
many developers bumped into these. Do-it-yourself solutions are left out of the comparison since those typically allow
extensive configuration of the limitations. These limitations are there for a reason; they make sure that the load
remains predictable for the provider, which allows them to provide lower latencies and better scaling. Azure is the
only provider that has no payload limits, which might explain why they are a bit behind performance-wise. Programmers
have struggled with these limitations and have found several creative
solutions around them. Some providers came up with their own solutions to improve the developer experience. For
example, Vercel builds upon AWS Lambda yet does not have the payload response limitation of Lambda.

Memory (MB)

Execution time

Payloads (MB)

Default

Max

Default

Max

Request

Response

Non-Edge offerings

Vercel Functions

3008

3008

10s-900s*2

10s-900s

5

No limit

Netlify Functions

1024

1024*3

10s*3

10s*3

6

6

IBM Cloud Functions

256

2048

1m

10m

5

5

Google cloud Functions

128

2048

NA

540s

10

10

Oracle Functions *4

128

1024

30s

120s

6

6

Azure Functions *5

NA

1536

5m

10m

No limit

No limit

AWS Lambda

128

3008

3s

15m

6*1

6*1

Alibaba Functions

?

?

?

600s

6*1

6*1

Edge offerings

Cloudflare Workers

128

128

No limit
10ms CPU*6

No limit
10ms CPU

?

?

EdgeEngine

128

128

15s
5ms CPU*6

15s
5ms CPU

?

?

AWS Lambda@Edge

128

128

30s

5s

50Mb

40 KB

Fly.io *7

NA

NA

No limit

No limit

No limit

No limit

*1: These are the specifics for synchronous requests. Asynchronous requests only provide 128Kb.

*2: Vercel’s execution time limits change depending on the plan and range between 10 seconds and 15 minutes.

*4: Oracle Functions also limit the amount of applications (10) and functions (20) per tenancy.

*5: Azure Functions have no specified limits, but the host machine’s limit is 1.5Gb.

*6: Cloudflare Workers and EdgeEngine are measured in CPU time. A function can use up to x ms of CPU time but can
run for a long time as long as it’s waiting and not using the CPU. Cloudflare has a limit of 10ms or 50 ms CPU
depending on the plan and no function duration limit while EdgeEngine has 5ms and a function limit of 15s and offers
custom plans via sales.

*7: Fly.io moved towards a global container runtime offering. The memory limits depend on the machine you select to run your container on, Besides of that there are no limits.

Pricing

The main four FaaS providers (Azure, AWS, Google, IBM) are very comparable.
Only Google comes out more expensive as can be verified using this online calculator. The new kids on the block are slightly pricier since they do
things quite differently. They abstract away a lot of work for you by automatically provisioning load balancers, or
making the deployment process much easier with local development tools, debugging tools, versioning, etc. These
providers aim to eliminate devops completely, so comparing their prices with bare-metal FaaS providers is like
comparing Heroku pricing with AWS. Pricing for custom setups is left out since they will basically depend on your own
infrastructure.

Free tier

Requests

GB seconds

Hours / Month

Non-Edge offerings

Netlify

125K *

NA

100 *

Vercel

NA

NA

20 *

IBM Cloud Functions

NA

400K

NA

Oracle Functions

2M

400K

NA

Alibaba Functions

1M

400K

GoogleFunctions

2M

400K

NA

AWS Lambda

1M

400K

NA

Azure Functions

1M

400K

NA

Edge offerings

Cloudflare Workers

NA

NA

NA

EdgeEngine

NA

NA

NA

Fly.io

$10/mo of service credit

Lambda@Edge

NA

NA

NA

*: Both Netlify as Vercel have a pricing model which starts with different plans, starting with a free plan. Other
plans start with a basic fee (which includes other services besides functions). Depending on the plan you choose,
these prices might differ, and there are also custom plans available.

Production pricing

Requests

GB seconds

Hours / Month

Non-Edge offerings

Netlify

$0.000038

NA

1h per 1000 requests

Vercel

NA

NA

$0.2 / h

IBM Cloud Functions

NA

$0.000017

NA

Oracle Functions

$0.0000002

$0.00001417

NA

Alibaba Functions

$0.0000002

$0.00001668

NA

GoogleFunctions

$0.0000004

$0.0000025

NA

AWS Lambda

$0.0000002

$0.00001667

NA

Azure Functions

$0.0000002

$0.000016

NA

Edge offerings

Cloudflare Workers

$0.0000005*2

NA

NA

EdgeEngine

$0.0000006*1

NA

NA

Fly.io

$0.000001015 / sec *3

NA

NA

Lambda@Edge

0,0000006

$0.00005001

NA

*1: EdgeEngine starts with a $10/month subscription which includes 15M requests.

*2: Cloudflare Workers starts with a $5/month subscription which includes 10M requests. Cloudflare Workers include
storage which is billed separately.

Languages

Most engineers prefer to program in a specific language. When moving to FaaS, this is not always possible since not
every language is supported by the providers. Additionally, some languages have significantly higher cold starts than
others. In theory, when deploying your own custom FaaS, any language is possible since they are built on top of Docker
and meant to be extensible. Of course, that might require a lot of work when providers such as IBM Cloud Functions and
Oracle Functions, who build upon OpenWhisk and Fn Project respectively, do not already have native support for all
languages. This table provides an overview of supported languages per service. Only the darker green ones can be
considered fully supported.

A

B

C

D

E

F

G

H

I

J

K

L

M

N

O

P

Q

R

1

OpenFaaS

Fn project

Fission

OpenWhisk

Kubel -ess

AWS Lambda

Google Functions

Azure Functions

IBM Cloud Functions

Alibaba Functions

Oracle Functions

Vercel Functions

Netlify Functions

Cloudflare Workers

EdgeEngine

Fly.io

Lambda@Edge

2

JS

✔︎

✔︎

✔︎

✔︎

✔︎

✔︎

✔︎

✔︎

✔︎

✔︎

✔︎

✔︎

✔︎

✔︎

✔︎

✔︎

✔︎

◉

Go

✔︎

✔︎

✔︎

✔︎

✔︎

✔︎

✔︎

✕

✔︎

✕

✔︎

✔︎

✔︎

◆

◆

✔︎

✕

4

Python

✔︎

✔︎

✔︎

✔︎

✔︎

✔︎

✔︎

▲

✔︎

✔︎

✔︎

✔︎

✕

◆

◆

✔︎

✔︎

5

Ruby

✔︎

✔︎

✔︎

✔︎

✔︎

✔︎

✕

✕

✔︎

✕

✔︎

✕

✕

◆

◆

✔︎

✕

6

Java

✔︎

◉

✔︎

✔︎

✔︎

✔︎

✕

▲

✔︎

✔︎

◉

✕

✕

◆

◆

✔︎

✕

7

C#

✔︎

◉

◉

✔︎

✔︎

✔︎

✕

✔︎

✔︎

✕

◉

✕

✕

◆

◆

✔︎

✕

8

C++

◉

◉

◉

◉

◉

◼︎

✕

✕

◉

✕

◉

✕

✕

◆

◆

✔︎

✕

9

Elixir

◉

◉

◉

◉

◉

◼︎

✕

✕

◉

✕

◉

✕

✕

◆

◆

✔︎

✕

10

Haskell

◉

◉

◉

◉

◉

◼︎

✕

✕

◉

✕

◉

✕

✕

◆

◆

✔︎

✕

11

Erlang

◉

◉

◉

◉

◉

◼︎

✕

✕

◉

✕

◉

✕

✕

◆

◆

✔︎

✕

12

Cobol

◉

◉

◉

◉

◉

◼︎

✕

✕

◉

✕

◉

✕

✕

◆

◆

✔︎

✕

13

Rust

◉

◉

◉

◉

◉

◼︎

✕

✕

◉

✕

◉

✕

✕

◆

◆

✔︎

✕

14

PHP

◉

◉

◉

◉

◉

◼︎

✕

▲

✔︎

✕

◉

✕

✕

◆

◆

✔︎

✕

15

Swift

◉

◉

✔︎

✔︎

◉

◼︎

✕

▲

✔︎

✕

◉

✕

✕

◆

◆

✔︎

✕

16

Bash

◉

◉

✔︎

◉

◉

◼︎

✕

▲

◉

✕

◉

✕

✕

◆

◆

✔︎

✕

17

Powersh

◉

◉

◉

◉

◉

✔︎

✕

▲

◉

✕

◉

✕

✕

◆

◆

✔︎

✕

◼︎ AWS Lambda runtimes in theory allow you to run any language but require extra work. At the moment, custom runtimes
were announced, they open sourced C++ and Rust runtimes, while other partners developed runtimes for Elixir, Elrang,
PHP, and Cobol.

▲ Azure has experimental support for these languages, but it is not advised to use them in production.

◉ Projects like OpenFaaS, Fn project, and OpenWhisk can in theory run any Docker container as a function. Therefore,
any language is supported, but it might require extra work. For example, OpenFaaS requires you to create OpenFaaS
templates, and OpenWhisk requires you to write something called Docker actions.

◆ via Web assembly so we assume at the moment that if it compiles to WebAssembly, it can be run.

Performance

Performance is typically the difference between an engaged customer and a bored client. FaaS performance is measured
in function call latency. This is very hard to compare across providers for two reasons:

Cold starts: When a function starts for the first time, it will respond slower than usual.

Different idle instance lifetimes: Each provider keeps its functions alive for a different amount
of time to mitigate cold starts.

Language and technology-specific: Cold starts and execution time are very
language-specific. The way the language is executed (compiled, interpreted, runtime, etc.) can have a
significant impact on the performance. For example, both EdgeEngine and Cloudflare Workers run JavaScript straight
on the V8 engine instead of on Node, which apparently decreases cold start latency significantly.

Between the three major providers, AWS is definitely still in the lead (1,2) in keeping cold starts low across all
languages. They also exhibit the most consistent performance. Azure is last with cold starts and overall performance
that is significantly worse than both Google and AWS. Nuweba has created a benchmarking website to compare these providers periodically.

Cloudflare Workers and EdgeEngine both provide a lower
cold start latency than regular FaaS providers thanks to the way they execute their functions. Both of these together
with Lambda@Edge also aim to reduce the latency that is perceived by
the caller by deploying functions in multiple locations and executing the functions as close to the caller as
possible. Since Cloudflare Workers currently have most locations, they can probably provide the lowest average latency
of the three edge providers. Cloudflare did a comparison between their own
workers, Lambda, and Lambda@Edge from which the results indicate that Cloudflare is several times faster.

There is also an ongoing independent serverless benchmark project
that is benchmarking both cold starts and execution time. It aims to extend the benchmarks to different providers than
the main three and already has benchmarks for AWS Lambda, Azure Functions, Google Cloud Functions, IBM Cloud Functions, and Cloudflare Workers. The results are in line with what we expect: AWS Lambda is consistently the fastest
for pure processing jobs yet is beaten in the domain of cold starts by Cloudflare. IBM Cloud Functions do very
well, but also have very high worst-case latencies. Note that the results are continuously updated and the images
provided here are snapshots. The results vary strongly from day-to-day.

The work of independent benchmarks is a great help to determine which functions are best suited for your problem.
Providers like Vercel and Netlify that rely on another provider will probably exhibit the same performance as the
underlying technology, which in both of these cases is AWS Lambda. At the time of writing, little work has been done
to benchmark the performance of other frameworks.

For do-it-yourself FaaS frameworks, keeping cold start latencies low, scaling, and distribution are your
responsibility. Different frameworks provide different configuration options for you to work with, but it will
probably be extremely difficult to compete with true serverless providers.

Overview

In a global overview, this is how they all compare.

A

B

C

D

E

F

G

H

I

J

K

L

M

2

AWS Lambda

Google Function

AzureFunction

Alibaba Function

IBMCloud Functions

Oracle Function

Vercel Function

NetlifyFunction

CloudflareWorkers

EdgeEngine

Fly.io

Lambda@Edge

3

Focus

general

general

general

general

general

general

Easy JAM,Edge

EasyJAM

Edge

Edge

Easy Edge

Edge

4

Speed

+++

++

+

?

++

?

?

?

+++

?

?

?

5

Free tier

1M reqs,400K Gb/s

2M reqs,400K Gb/s

1M reqs,400K Gb/s

1M reqs,400K Gb/s

Unlimited reqs400K Gb/s

2M reqs,400K Gb/s

20 hours /month

100 hours /month

None

None

$25

None

6

Pricing

$0.0000002 / request

$0.0000025 Gb/s

$0.0000004 / request

$0.00001667 Gb/s

$0.0000002 / request

$0.000016Gb/s

$0.0000004 / request

$0.00001668 Gb/s

$0.000017Gb/s

$0.0000002 / request

$0.00001417Gb/s

$0.2 / h

$0.000038 /request

$0.0000005/request

$0.0000006/request

$0.000001015 / sec

$0.00005001/request

7

Languages

JSGo,PythonRubyJavaC#PowerSh

JS,Go,Python

JS,C#

JSPythonJavaPHP

JS,Go,Python,Ruby,Java,C#

JS,Go,Python,Ruby

JS,Go,Python

JS,go

JSWebassembly

JSWebassembly

JS

JSPython

8

Memory Limits

128-3008

128-2048

?-1536

128-2048

128-1024

3008-3008

1024-1024

128-128

128-128

128-128

128-128

9

Execution time Limits

15m

9m

10m

10m

10m

2m

15m

10s

No limit 10ms CPU

15s

?

5s

10

Payload request

6Mb

10Mb

No limit?

6Mb

5Mb

6Mb

5Mb

6Mb

?

?

?

50Mb

11

Payloadresponse

6Mb

10Mb

No limit?

6Mb

5Mb

6Mb

No limit?

6Mb

?

?

?

40Kb

12

Conclusion

A while ago, choosing a FaaS provider was a relatively easy task. Today, we are spoiled with diversity since FaaS
providers are springing up like mushrooms. With such a wide range of providers, it becomes harder and harder to follow
up on what exists and how they differ. Realizing that some new offerings have an entirely different focus
(easy-of-use, edge, do-it-yourself) already brings you one step closer to choosing the right provider. In an attempt
to make your choice more comfortable, we have researched and compared various providers on topics such as focus,
limitations, pricing, languages, and performance. We hope that this becomes a basis for further discussion as such a
comparison is ideally a collaborative effort.

If you enjoyed our blog, and want to work on systems and challenges related to serverless databases, GraphQL, and JAMstack, Fauna is hiring!