I want to take a slightly different cut at the topic of performance. While
it’s a messy topic, I’m going to try to lay out some of the additional questions
that developers should be asking a Startup Founder around the performance
requirements of the application.

To get us started and to grossly oversimplify performance, conceptually we
can think about the system as consisting of the following elements that I’ll
refer to throughout the post.

Requests. We get a set of requests for our system to do something –
generally from users or external systems.

Compute. Our system must access our data, possibly 3rd party services, do
some calculation and then get back to the user or the other system with our
response.

Response. The pages or API response we provide back.

Application Characteristics

Any discussion of performance starts by understanding what the software does,
how it is used, and what it interacts with. A developer should find out:

What are the different types of users? What do they do?

Is SEO important? – this is really another type of user – a crawler

Are we providing an API to other systems? What are the characteristics of
how these are used? - again, this is like a user type.

Are there any time-based operations? Overnight calculations?

Are 3rd party services used? What are the characteristics of how they are
used? What are their performance characteristics?

How many of each type of user are there? How many might be using the
application at the same time (concurrent users)? Will there be spikes of
concurrent usage?

What data is used in the application? How big is the data set? Are there
complex aspects to the data?

What computations / algorithms are part of the application? Are any of the
calculations done often? Are any of the calculations complex?

Are there any aspects that have specific performance needs? For example,
are you providing a stats service that needs frequent, fast updates?

Response Time

Once we understand the overall characteristics of the application, then we
want to drill down on some specific performance characteristics. We generally
start with response time needs because, in many ways, this is ultimately the
measure of performance. If you think about our system picture above, response
time is roughly the time it takes to get our page or API call back from the
system.

It’s well documented that response time has significant business impact:

The impact is quite real. But as with most things in tech, the picture is
far more complicated than that. Consider two different types of systems:

eCommerce or Content web site. These will have many individual web pages,
with specific URLs, optimized for SEO. Each page needs fast response time (both
time to first byte and total load time). Pages may not have much dynamic
content on the page. There may be lots of pages.

Web Application such as Web Mail or a gated Social network. The content is
not used for SEO so response time characteristics may be quite different. If
the initial load time of the web application was 10 seconds but bringing up an
individual email took less than 1s that’s likely an okay characteristic.
Technically, this may open the door to a single-page application (SPA). These
often often have a relatively longer time to load and then has really good
performance once you are “in the application.”

Of course, response time is quite a bit more complex than this. You will be
looking at aspects like:

Time to first byte (TTFB) vs. Load time

API calls

Global delivery?

Mobile delivery possibly with slow connections?

As a startup founder, you need to think about the characteristics of your
solution and what you need from a response time standpoint.

Request Volume

Assuming we know what our system needs to produce (the right side of the
picture) and how fast (response time), then the next big question is really how
much? We want to find out what requests the application gets (left side of
picture) and how often these come in. This is generally turned into a Requests
per Second number.

Most of the time we will start by asking about Concurrent Users – and this is
generally the number that startup founders are thinking about when they talk
about scalability. Concurrent users are the number that are on your web site or
web application at the same time. Of course we need to combine number of
concurrent users with what the users are doing in order to have more of a
picture of what this means.

For example, let’s assume this is a content site. For human users, they
request a page with content, likely the content page is relatively simple, the
user reads/scans the page for a little bit, they decide to click something else
which requests a new page. This may take 10 seconds. So some quick math:

Each user generates 0.1 requests per second

1,000 concurrent users generate 100 requests per second

Those are really interesting numbers for a technical person. Of course, this
gets much more complicated. A developer will want to drill down on:

Different types of users?

Different use cases?

Traffic spikes? TV Coverage? Real-time events?

API Usage?

Growth rates?

This will give us a clearer picture of Request Volume for different kinds of
requests that our system needs to process.

Complexity

Now we know the volume we need to satisfy coming in on the left and the
response time required on the right. The middle is what the system needs to do
in order to respond to that volume of requests within that timeframe.

Developers will want to explore with a startup founder where there may be
complexity in the system. We want to do this for two reasons: (1) how complex
is the software we need to build – complexity generally means more time/cost to
build, and (2) how long will it take for the system to calculate responses. I’m
only going to focus on the second aspect – understanding complexity as it
relates to performance. And really I’m only going to scratch the surface here
as complexity is something that a startup founder and a Technical Advisor would need to explore together.

Computation Complexity – What do we need to compute? What are some of the
more complex aspects of system calculation? Natural language processing?
Matching algorithms? Complex reports? Are there widely varied use cases with
different performance characteristics? Any blocking operations?

Data Complexity – What data are we dealing with? How big is the data set?
What are the largest number of a single type of entity? Are there aspects that
need to be pre-computed? Any time series data? Any logging/auditing data?

3rd Party System Complexity – What are the characteristics of the 3rd party
systems? What will happen when they are slow or non-responsive? What happens
when they return poor quality results?

Last Thoughts

Yikes, that turned into a lot more than I was originally thinking as I
started this post. Hopefully the core model makes sense. As a startup founder
you need to think about the characteristics of your application and then think
about the Volume, Complexity and Response Time requirements. For some
applications, it will be relatively straight forward to think about the
technical performance requirements for your startup. However, in many cases,
this is a place where you really should be talking with a technical advisor or
reaching out to get Free Startup CTO Consulting in order to understand what you
need.

About Me

Dr. Tony Karrer works as a part-time CTO for startups and midsize software companies - helping them get product out the door and turn around technology issues. He is considered one of the top technologists in eLearning and is known for working with numerous startups including being the original CTO for eHarmony for its first four years. Dr. Karrer taught Computer Science for eleven years. He has also worked on projects for many Fortune 500 companies including Credit
Suisse, Royal Bank of Canada, Citibank, Lexus, Microsoft, Nissan,
Universal, IBM, Hewlett-Packard, Sun Microsystems, Fidelity
Investments, Symbol Technologies and SHL Systemhouse. Dr. Karrer was
valedictorian at Loyola Marymount University, attended the University
of Southern California as a Tau Beta Pi fellow, one of the top 30
engineers in the nation, and received a M.S. and Ph.D. in Computer
Science. He is a frequent speaker at industry and academic events.