Assessing Fund Performance:

1.
Assessing Fund Performance:
Using Benchmarks in Venture Capital
Venture Capital Update
2008
written by: The goal of benchmarking is to measure venture capital industry benchmarks.
comparative performance. When it We explain the metrics and methods
Bronwyn Dylla Bailey comes to cars and other manufactured and review the benefits and limitations
Research Director
products, measurements are precise of benchmarks commonly used in the
650.855.3021
bbailey@svb.com and benchmarks are consistently industry. It is our hope that a better
applied. As a result, benchmarks create understanding of benchmarking
Aaron Gershenberg value by identifying performance gaps across the investment community
Managing Partner and enabling better decision making can lead to improved means of
650.855.3011 based on facts. Venture benchmarks, developing and gaining value from
agershenberg@svb.com however, are a different story. the benchmarks.
As a way to compare the performance WHY WE NEED BENCHMARKS
of venture capital funds over time,
available industry benchmarks Making an investment in venture
can be inconsistent and confusing. capital is a long-term commitment,
Not surprisingly, LPs often seek with 10 years being the typical lifespan
independent verification of these of a fund. During this period, LPs
claims. To assess the performance receive quarterly financial reports on
of our own funds, SVB Capital capital calls and distributions related
continuously examines the best to their investment. However, they
methods of using these statistics. also need to understand how their
investment is performing while the
View the Fourth Quarter
2007 U.S. Private Equity In this issue of Venture Capital Update, capital is put to work during the
Snapshot SVB Capital shares our findings about “J-curve” and before the fund’s
ASSESSING FUND PERFORMANCE VENTURE CAPITAL UPDATE 1
2008

2.
Venture Capital Update
portfolio is completely realized. IRR provides an effective rate of performance because they manage
Financial statements alone will not return based on cash flows and current a portfolio that includes a mix of
provide this perspective, so LPs valuations of the fund portfolio, while public and private investments.
typically turn to benchmarks. DPI shows the realized portion of the Consequently, IRR reported
portfolio that was distributed to the as a percentage provides an easy
In addition to gauging the returns LP as a multiple of the contributed comparison to return percentages
they might expect over time, LPs capital. By comparison, TVPI on public investments, even though
also require a way to compare the provides a multiple value on the IRR percentages are not completely
performance of investments across entire portfolio—both distributed comparable.2 LPs will often look for
their portfolio. This is true even capital and the net asset value of the 400 to 600 basis points over a public
when investments are in different portfolio.1 benchmark to justify the illiquid and
asset classes such as public equity long-term nature and risk profile
and private equity, and regardless Which of these metrics is the best of VC investing. DPI provides a
of whether the LP is a private assessment of fund performance? The clear metric of the actual multiple
individual, endowment, private or short answer is, “it depends.” Many of cash invested which has been
public pension fund. LPs rely on IRR measurements of received by an investor, and TVPI
provides a metric that accounts for
But benchmarks are also an important The performance of a venture capital
potential returns that are the result
indicator for venture capitalists fund can be calculated via at least one of increased valuations of portfolio
(VCs). Sizing up the performance of of the following metrics: companies as they approach exit.
a fund in the middle of its life cycle Given this difference, many LPs
is key to assessing how portfolio IRR: The annualized effective return rely on TVPI earlier in the life of a
companies are performing relative to rate which can be earned on the fund and DPI towards the end. In
the market. contributed (invested) capital, i.e. the contrast to IRR, TVPI and DPI do
yield on the investment. not account for the time it takes to
PERFORMANCE METRICS: produce these gains.
APPLES AND ORANGES DPI: The ratio of cumulative
distributions to limited partners Despite the shortfalls, the three
When assessing the return-on- divided by the amount of capital metrics have become the standard for
investment performance of a venture contributed by the limited partners. comparison. In our experience, we
fund, three different metrics are have found that LPs rely on a
typically used: TVPI: The sum of cumulative combination of all three metrics
• internal rate of return (IRR) distributions to limited partners and to assess the performance of their
• distributions to paid-in capital the net asset value of their investment, investments, with some favoring
(DPI) divided by the capital contributed by one over the other, in part, due to
• total value to paid-in capital the limited partners. the preference of their board and
(TVPI) their specific type of investment.
ASSESSING FUND PERFORMANCE VENTURE CAPITAL UPDATE 2
2008

3.
Venture Capital Update
fund a: returns spread evenly over 10 years fund b: big return early in fund
DPI TVPI IRR
7.0 7.0
2400% 2400%
6.0 6.0
2000% 2000%
5.0 5.0
DPI & TVPI
DPI & TVPI
1600% 1600%
4.0 4.0
IRR
IRR
3.0 1200% 3.0 1200%
2.0 800% 2.0 800%
1.0 400% 1.0 400%
- 0% - 0%
00
01
02
04
05
06
07
08
09
10
11
00
01
02
04
05
06
07
08
09
10
11
03
03
20
20
20
20
20
20
20
20
20
20
20
20
20
20
20
20
20
20
20
20
20
20
20
20
0/
0/
1/
1/
1/
0/
9/
9/
1/
1/
1/
0/
9/
9/
1/
1/
0/
0/
9/
9/
0/
0/
9/
9/
/3
/3
/3
/3
/3
/2
/2
/3
/3
/3
/2
/2
/3
/3
/3
/2
/2
/3
/2
/2
/3
/3
/3
/3
12
12
12
12
12
12
12
12
12
12
12
12
12
12
12
12
12
12
12
12
12
12
12
12
Cost: ($100m) IRR = 14% Cost: ($100m) IRR = 2191%
Returns: $200m TVPI = 2.0x Returns: $110m TVPI = 1.1x
Gain: $100m Gain: $10m
All three metrics can result in a biased multiples on returned capital— requiring large amounts of capital
assessment of fund performance due that is, more money back into the very early in a company’s life cycle,
to the way each is calculated. For investor’s pocket. This example also and requiring longer investment
instance, the calculation of IRR is shows that big returns during the periods, may inherently generate
greatly influenced by the timing of first few years of a fund’s life can lower IRRs. Larger capital calls
returns in a fund, or more specifically, lead to misleadingly high TVPI and would occur earlier and returns
short holding periods.3 The example would be realized later in the life
DPI multiples early in the fund. In
above shows two funds of identical
this example, Fund B showed TVPI of this type of fund as compared
size and capital call timing. Fund A
and DPI figures above 5.0x in the to other funds. Companies in life
provides a steady return of 2.0x to its
first year in the life of the fund, but sciences may fit this profile, while
investors during the last four years
the fund ultimately returned only quicker exits might come from Web
of the fund with an ending IRR of
1.1x at termination. 2.0 companies.
14 percent. Fund B returns only
1.1x, but with an IRR of more than
2000 percent due to the large returns The built-in bias of the performance Likewise, later-stage investments
early in the fund’s life, soon after the metrics may have greater implications potentially would generate returns
investment was made. for certain types of funds, particularly after a shorter time period than early-
funds that focus on certain stages or stage funds, with a potentially higher
This example shows that funds with sectors. For instance, funds with IRR due to the timing of the returns,
lower IRRs can still provide higher investments concentrated in sectors assuming equal performance.
ASSESSING FUND PERFORMANCE VENTURE CAPITAL UPDATE 3
2008

4.
Venture Capital Update
OVERVIEW OF INDUSTRY Let’s explore each one of these Because the data are gathered from
BENCHMARK SOURCES differences in more detail. public sources, Preqin publishes the
performance metrics for specific funds
The most commonly used industry 1. The benchmarks use different and firms. That is, it does not keep
benchmarks are published by methodologies for data collection the fund or firm name confidential
Cambridge Associates and Thomson for performance on individual funds.
Reuters Venture Economics. Performance metrics vary widely Cambridge Associates and Thomson
Cambridge Associates is a consulting from one benchmarking source to Reuters aggregate fund performance
firm that provides advisory services another. One factor is the different information and do not identify fund
to institutional investors and in doing methodologies for collecting data or firm names. Preqin advertises
so, has access to financial information from the funds. Cambridge Associates that its data have less selection bias
for a large number of funds. Thomson collects financial information from than samples collected via surveys or
Reuters (formerly Thomson Financial) its clients’ investments as well as by client investments because Preqin’s
publishes a range of financial news soliciting information from managers,
and information, such as Private information would not omit better
which it aggregates into its database for funds or worse-performing funds or
Equity Week and the Venture Capital
calculating performance benchmarks. be skewed upwards by institutional
Journal. A third benchmarking source,
Thomson Reuters Venture Economics
Private Equity Intelligence (known as clients’ investment picks. However,
uses surveys sent to private equity
“Preqin”), creates benchmarks using some in the industry assume that
and venture funds relying on self-
its Performance Analyst database of since Preqin gathers data from funds
reporting. These surveys are not
fund financials. Preqin also provides subject to disclosure, these investors
audited, but the information collected
access to separate databases of funds cannot access the best performing
reveals cash flow information. Both
in the market and limited partner funds and therefore, Preqin’s results
organizations collect this information
information. will be skewed downward.
on a confidential basis.
These organizations are more 2. The benchmarks use different
different than they are similar, not By contrast, Preqin collects data on
data samples
only in their business structure, but in fund performance based on public
how they gather, analyze and report data sources, typically reports
from pension funds and other One reason why the performance
benchmarks. Specifically:
institutions that must provide their benchmarks from each of the
financial performance reports as providers are so different is because
1. The benchmarks use different
mandated by the U.S. Freedom of they use different samples of funds
methodologies for data
Information Act (FOIA) or similar for their calculations. Simply put,
collection
legislation in foreign countries. These different samples of funds yield
2. The benchmarks use different different benchmarks. The graph
organizations report performance,
data samples below compares the number of funds
rather than cash flows, which form
3. The benchmarks provide the basis of calculations by Thomson per vintage year from 1995 to 2007 for
different performance results Reuters and Cambridge Associates. each of the benchmark providers.
ASSESSING FUND PERFORMANCE VENTURE CAPITAL UPDATE 4
2008

5.
Venture Capital Update
venture. However, it is known that
number of u.s. venture funds sampled by vintage year these samples are small percentages of
funds in the U.S. market. The graph
Cambridge Associates Thomson Reuters Preqin
below shows the sample size for
180 162 each of the benchmark providers as a
160
percentage of funds invested.
140 125
118
Number of funds
120
100
The available sample sizes seem small
82 109
80 73 given the perceived variance in fund
57
60 49 43 77 72 56
58 55 50 performance, but this is beyond the
62 35 37
40 35 54 30 28 33
44 34
control of the organizations providing
36 44 17
20 35 38 27 the benchmarks. In one survey of
25 25 19 16 23 17 25 11
0 private equity firm CFOs, almost 40
1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007
percent of respondents stated that they
Vintage Year
did not send financial information
Sources: Cambridge Associates, Thomson Reuters, and Private Equity Intelligence. Cambridge Associates
to Thomson Reuters because there
and Thomson Reuters data are as of December 31, 2007; Preqin data are as of various dates but are the was “no reason to do so.”5 In fact,
most recent obtained. Cambridge Associates data were provided at no charge. there is little incentive for funds to
While Cambridge Associates typically complete and return surveys of their
statistically significant. Because there
uses the largest sample size to calculate performance, particularly if the fund
is not a precise estimate of variation
benchmarks for almost all of the is one of the best or one of the worst
of funds’ performance, it’s difficult to
vintage years during 1995 to 2007, performing.
estimate an accurate sample size for
it is questionable whether even its
sample size of funds per vintage year sample size as a percentage of active u.s. venture funds
is large enough to provide efficient and
unbiased estimators of performance.4 Cambridge Associates Thomson Reuters Preqin
In other words, does a summary 33
35% 33
statistic of performance based on these 32 31
29 29
Percentage of Active Funds
30% 28 28
sample sizes reflect actual performance 27 27 27
26
29
of venture funds in the market? 25%
26 23
22 18 24
20% 21 17
21 15 16 18
In statistics, the amount of variance 15% 18 13
16 15 14
in the population must be known 15 14
13
14 15 14 13 9
10% 12
or estimated in order to determine 6
5%
the appropriate sample size. The
performance of venture funds is 0%
1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007
known to have a large amount of Vintage Year
variability because the dispersion of
returns is large. And the greater the Sources: Cambridge Associates, Thomson Reuters, and Private Equity Intelligence. Cambridge Associates
and Thomson Reuters data are as of December 31, 2007; Preqin data are as of various dates but are the
variation, the larger the sample size most recent obtained. The number of funds in each vintage year is the number of active funds based on
required for the sample’s metrics to be Thomson Reuters Fund Statistics Report. Cambridge Associates data were provided at no charge.
ASSESSING FUND PERFORMANCE VENTURE CAPITAL UPDATE 5
2008

6.
Venture Capital Update
Do the differences in aggregated
comparison of investment horizon benchmarks
performance indicate that these sources
contain completely different collections
Thomson Reuters Cambridge Associates
40% of funds? An examination of the
35% sampling distributions and the sample
means, or averages, would provide
30%
a definitive answer to this question.
Pooled IRR (%)
25%
However, the data for individual funds
20%
in Cambridge Associates and Thomson
15% Reuters’s samples are not available
10% to conduct these statistical tests. As
5% a proxy, box and whisker plots help
0% show the quartile ranges of funds and
10-Year Venture IRR 5-Year Venture IRR 3-Year Venture IRR 1-Year Venture IRR the best and worst performing funds.
Performance Performance Performance Performance
The chart below show the maximum,
Sources: Cambridge Associates and Thomson Reuters. Note that Preqin does not provide cumulative top quartile, median, lower quartile
benchmarks over specific time horizons. Pooled IRR is calculated based on cash flows of all funds regardless and minimum fund performance for
of vintage year during the specified time horizons. All data are as of December 31, 2007. Cambridge
Associates data were provided at no charge.
vintage year 2000.6 The top quartile
and lower quartile provide the top and
3. The benchmarks provide different Large differences also remain in the bottom edges of the box; the median
performance results three-year pooled IRR performance is the line in the middle; the minimum
benchmarks. and maximum are dots connected by
While each benchmarking source extended lines.
purports to report on the performance
of the industry, there is a large variation comparison of irr ranges for vintage 2000
in performance metrics among the
Lower Quartile Minimum Median Maximum Upper Quartile
three providers. The bar graph above 40%
compares pooled IRR performance
metrics between Cambridge Associates 20%
and Thomson Reuters. Preqin does
0%
not provide benchmark metrics over
IRR (%)
these time horizons; rather the data -20%
are provided by fund, firm or vintage
year. Note that Cambridge Associates’ -40%
10-year pooled IRR figure is almost
-60%
double the same metric published
by Thomson Reuters. Given that the -80%
pooled 10-year IRR metric is more Cambridge Associates Thomson Reuters Preqin
stable than, for instance, a short-term
Sources: Cambridge Associates, Thomson Reuters, and Private Equity Intelligence. Cambridge Associates
one-year metric, the large difference in and Thomson Reuters data are as of December 31, 2007; Preqin data are as of various dates but are the
the long-term benchmark is surprising. most recent obtained. Cambridge Associates data were provided at no charge.
ASSESSING FUND PERFORMANCE VENTURE CAPITAL UPDATE 6
2008

7.
Venture Capital Update
This simple analysis confirms the fund in samples from Cambridge OVERCOMING BENCHMARK
wide range of performance across Associates, Thomson Reuters and LIMITATIONS
venture capital funds—not only Preqin. The difference in IRR metrics
within samples, but also across of the sources’ top quartile funds was While benchmarks can provide a
different benchmarking sources. This narrow (2 percent) in 2001 and wide quick comparison of one investment
disparity in performance between the (12 percent) in 2005. The graph also to the performance of another in
best and worst funds is exceptionally shows that no one benchmark has an the same asset class, many LPs
large—particularly for Cambridge upper quartile fund performance that invest in venture capital and private
Associates’ funds in 2000—but the is consistently higher or lower than equity to add diversification to their
performance of these funds may be the other benchmarks; moreover, the portfolio and to provide returns
outliers compared to other funds benchmarks do not trend together. that are not correlated to public
in each sample. Nonetheless, the
benchmarking sources show ranges upper quartile comparison
of more than 10 percentage points
between the upper quartile and lower Cambridge Associates Thomson Reuters Preqin
30%
quartile IRRs and almost 20 percentage
25% 24
points’ difference in Preqin’s sample.
21
20%
Pooled IRR (%)
16
For vintage year 2000, the median 15%
11
13 14 12
10
fund performance is similar across 10% 11 12 11 12
10
the three benchmarking sources, 9
5% 7 5
5
although the IRR is positive according 0% 3 4
to Preqin and negative according to -3
-5%
Cambridge Associates and Thomson 2000 2001 2002 2003 2004 2005 2006
Reuters.7
Sources: Cambridge Associates, Thomson Reuters, and Private Equity Intelligence. Cambridge Associates
and Thomson Reuters data are as of December 31, 2007; Preqin data are as of various dates but are
Many in the venture industry would the most recent obtained. Preqin does not provide IRR benchmarks for vintage 2007 funds. Cambridge
Associates data were provided at no charge. Note: IRR performance during the first three years of a fund is
argue that a fund’s performance typically considered not meaningful.
must place it in the top quartile
in order to achieve attractive, risk-
adjusted returns over time. The box In the process of benchmarking performance, LPs must decide the objective of an
and whisker plots above show how investment. Which is it?
high the performance can be for • Provide a return commensurate with the added risk and illiquidity of the investment
some of the funds in the top quartile • Provide more dollars back to the fund
(represented by the extended lines • Outperform public investments by a certain margin
on top of the boxes). The graph Determining the primary goal of the investment will help to guide LPs to find the appropriate
below shows the variation in the benchmarking tool.
IRR performance of the top quartile
ASSESSING FUND PERFORMANCE VENTURE CAPITAL UPDATE 7
2008

8.
Venture Capital Update
markets. These LPs often compare hard to collect accurate financial information to benchmarking
the IRR of the venture portion of data on private investments. It’s organizations, SVB Capital believes
their portfolio to the performance also difficult to report consistently that the venture industry as a whole
of public investments, with an on performance due to the metrics, should have an incentive to create
expectation that the venture portion the sample sizes and the collection more credible and statistically reliable
will return a certain level higher methodologies. performance metrics. With more
than public market investments. The accurate benchmarks, the venture
logic of this assessment is based on With clear shortcomings and industry could assess more fully
risk and reward. Venture investing inconsistencies in industry how funds are performing, especially
presents greater risks to an investor benchmarks, how can investors as compared to other asset classes,
than investing in public markets, in assess the performance of their funds? and communicate these results with
part, because it is a long-term and Given the long-term nature of the current and potential investors.
relatively illiquid investment; likewise, investment and the lack of access to
investors expect greater returns from information on returns in the private Today it’s commonplace to study
their venture investments. market, accurately benchmarking automotive industry benchmarks
venture capital remains elusive. that yield meaningful insights as a
The business of benchmarking basis for decisions. Tomorrow it’s
venture capital funds has many While individual funds may have little possible we will be able to say the
complications simply because it is incentive to contribute their financial same about venture capital.
Recognizing the limitations of venture capital benchmarking for assessing performance, SVB Capital recommends supplementing
benchmark analysis with other information. Consider the following:
• Gain a better understanding of portfolio companies and the return potential of the active portfolio. Annual meetings can typically be
the place to obtain this information. It is widely known that one “home run” in a venture capital portfolio can move the fund into top-tier
territory. Returns to the top-tier venture capital funds are typically driven by a few deals.
• Become part of the conversation. Those closest to the business of the fund have good information and instincts about current and
future performance. Discuss the performance of the fund with the fund managers and learn the details of the companies in the portfolios
that drive—or drag—performance.
• Look at the track record of individual venture investors, many of whom have made previous investments at other funds. The performance
of past investments—including which sectors provided the returns—might help to inform expected performance.
ASSESSING FUND PERFORMANCE VENTURE CAPITAL UPDATE 8
2008

9.
Venture Capital Update
TELL US WHAT YOU THINK
Send your comments and suggestions for topics to Bronwyn Bailey at bbailey@svb.com.
*This update is for informational purposes only and is not a solicitation or recommendation that any particular investor
should invest in any particular industry, security, or fund.
This material, including without limitation to the statistical information herein, is provided for informational purposes
only. The material is based in part on information from third-party sources that we believe to be reliable, but which
have not been independently verified by us and for this reason we do not represent that the information is accurate
or complete. The information should not be viewed as tax, investment, legal or other advice nor is it to be relied on in
making an investment or other decision. You should obtain relevant and specific professional advice before making any
investment decision. Nothing relating to the material should be construed as a solicitation, offer or recommendation to
acquire or dispose of any investment or to engage in any other transaction.
1
Net Asset Value is the market value of the portfolio plus any cash held by the fund.
2 See Austin M. Long and Craig J. Nickels, “A Method for Comparing Private Market Internal Rates of Return to Public Market Index Returns.” Manuscript. The
University of Texas System, August 28, 1995.
3
The IRR calculation assumes that distributed capital is reinvested at the same IRR over the life of the fund, when in fact, investors may not find similar investment
opportunities for each distribution. See Oliver Gottschalg and Ludovic Phalippou, “The Truth about Private Equity Performance,” Harvard Business Review,
December 2007 for this analysis. For a detailed discussion on the benefits and drawbacks of using IRR as a performance metric, see Paul Gompers and Josh Lerner,
“Assessing the Performance of Private Equity Funds.” Manuscript. Harvard Business School, January 2003.
4 The game of darts can be used as an analogy for the quality of a sample statistic. The darts of an efficient and unbiased player would land clustered closely together
on the bulls’ eye of the target. The darts of a less efficient player would land scattered around the dartboard, and the darts of a biased player would be tightly
clustered outside the bulls’ eye.
5 Results from an informal survey conducted by Thomson Reuters (formerly Thomson Financial) presented at the Private Equity CFO Conference, July 2007.
Respondents were attendees at the conference.
6
Vintage year 2000 was chosen due to the large number of funds in each sample, which would provide a more conservative estimate of variation in fund performance.
This analysis is limited to IRR performance because Cambridge Associates does not publish quartile ranges for DPI and TVPI calculations.
7
This finding for vintage year 2000 does not support the notion that public institutions, Preqin’s data source, have problems accessing better performing funds.
ASSESSING FUND PERFORMANCE VENTURE CAPITAL UPDATE 9
2008