Discovering the quality of portfolio decisions

The portfolio

We explore a particular portfolio during 2007. It invests in S&P 500 stocks and starts the year with a value of $10 million. Initially there are 50 names in the portfolio. It also ends the year with 50 names but has up to 53 names during the year.

The constraints on the portfolio are:

long-only

50 to 60 names

no asset may have more than 4% weight when the portfolio is changed

The portfolio was changed 10 times during the year. Figure 1 shows the cumulative amount traded (buys plus sells) throughout the year.

Figure 1: Cumulative turnover (buys plus sells) in millions of dollars. The amount and timing of the changes are irregular. For simplicity there is no cashflow, but that is only a minor complication for the analysis to come.

Figure 2 shows the value of the portfolio throughout 2007.

Figure 2: Value (in millions of dollars) of the portfolio during 2007.

Decisions

Performance analysis is — or should be — about decisions.

The current analysis makes two changes that overcome weaknesses of traditional performance analyses:

Decisions and their effects are broken into discrete time frames

The results of decisions are compared to those of alternative decisions that could have been made

Performance measurement for 2007

Traditionally our portfolio would be compared to the S&P 500 index. Figure 3 compares the cumulative returns through 2007 of the portfolio and the index.

Figure 3: Cumulative returns during 2007 for the portfolio (blue) and the S&P 500 (black). The portfolio outperforms the index. That’s nice, but it doesn’t really inform us about the decisions that were made with the portfolio. These two portfolios have one decision in common: the universe of assets. All of the other portfolio decisions are different. The relevance of the index to portfolio decisions is at best vague.

We can get a better handle on the portfolio decisions by breaking them into two pieces:

decisions made during 2007

decisions made prior to 2007

We’ll look at the second of these in a later section. For now we concentrate on the decisions that were made during 2007.

There is an important — and often neglected — constraint facing the fund manager at the start of 2007: the portfolio as it exists at that point. One vision of the fund manager’s job is to optimally trade away from where the portfolio starts.

Figure 4 is a more informative comparison: the cumulative returns of the portfolio, and the cumulative returns if no trading had taken place during 2007.

Figure 4: Cumulative returns during 2007 for the portfolio (blue) and the portfolio as of the start of 2007 (green). This plot gives us a minor amount of information about a pertinent question. The question is:

Question 1: How good — during 2007 — were the decisions made in 2007?

Figure 4 shows two possible choices for the fund manager during 2007: the one actually taken and the possibility of not trading at all. There were many choices available and it is those other choices that are the heart of Question 1. Question 1 can be rephrased as:

Question 1 (alternate): How good — during 2007 — were the decisions made in 2007 relative to the set of choices that were available?

We can get a set of representative alternative choices by imitating what happened to the portfolio during the year. To create one of these choices:

Figure 5 compares the value of the actual portfolio to the random alternatives.

Figure 5: Value (in millions of dollars) during 2007 of: the portfolio (blue), the random benchmarks (gold) and no trading (green). Now we have some specific evidence with which to answer Question 1. We know that as of the end of the year the decisions were either useful or lucky. But that characterization was not true throughout the whole year. Figure 6 shows the fraction of random benchmarks that outperformed the portfolio for each day during the year.

Figure 6: Performance percentiles of 2007 decisions during 2007. In Figure 6 we see that changes to the portfolio were a couple of months too early, and then there were two instances where the strategy lost power.

Performance attribution for 2007

The Brinson model is a handy way to attribute performance to members of a category — sectors for instance. Wikipedia has a simple example and Morningstar gives a fuller explanation.

We can do an attribution to sectors for our portfolio for the whole of 2007. Here are returns (in percent) by sector.

Port ret

Bench ret

Consumer Discretionary

7.33

1.36

Consumer Staples

-1.13

4.98

Energy

31.20

30.52

Financials

8.41

7.96

Health Care

27.33

7.80

Industrials

17.30

22.47

Information Technology

14.26

19.65

Materials

84.71

48.53

Telecommunications Services

0.00

0.00

Utilities

3.92

5.96

Total

20.79

13.64

The weights (in percent) by sector are:

Port wt

Bench wt

Consumer Discretionary

29.56

23.82

Consumer Staples

11.26

14.68

Energy

7.01

7.65

Financials

4.43

6.18

Health Care

11.61

11.71

Industrials

9.63

14.61

Information Technology

10.57

9.89

Materials

11.42

6.89

Telecommunications Services

0.00

0.00

Utilities

4.51

4.57

Total

100.00

100.00

The actual attribution analysis (in percent) is:

Selection

Allocation

Interaction

Consumer Discretionary

1.42

0.08

0.34

Consumer Staples

-0.90

-0.17

0.21

Energy

0.05

-0.20

-0.00

Financials

0.03

-0.14

-0.01

Health Care

2.29

-0.01

-0.02

Industrials

-0.76

-1.12

0.26

Information Technology

-0.53

0.13

-0.04

Materials

2.49

2.20

1.64

Telecommunications Services

0.00

0.00

0.00

Utilities

-0.09

-0.00

0.00

Total

4.00

0.78

2.38

Traditionally the benchmark would be an index. In this case the benchmark is “no trading through the year”. That is, the benchmark portfolio is the initial portfolio that the fund held at the beginning of the year.

A summary of the analysis seems to be that the trading during the year was good at asset selection. Both asset allocation and interaction were also positive.

While not trading at all is a particularly interesting alternative to what the fund manager actually did, it is just one alternative. We are not getting a picture of how good the asset selection and allocation are relative to all the alternative choices the fund manager had.

We can do this with graphs that show how these change through the year. In all of the graphs shown here the starting time for each point is the first time in the plot. So at the start of February the attribution is for a one-month period, and at the start of July the attribution is for a six-month period.

Asset selection

Figure 7 shows the selection effect for the portfolio against each of the random benchmarks. This is asking if the portfolio returns within a sector were better than the benchmark returns within that sector.

Figure 7: Whole portfolio selection against the random benchmarks during 2007. We see a strong selection effect in the last part of the year. We also see substantial spread — 7% rather than 4% at the end of the year could well be a matter of luck rather than skill.

Graphs can also be produced for individual sectors. Figures 8 through 10 are examples.

Figure 8: Asset selection against the random benchmarks during 2007 for Health Care.

Figure 9: Asset selection against the random benchmarks during 2007 for Industrials.

Figure 10: Asset selection against the random benchmarks during 2007 for Materials.

Asset allocation

The asset allocation effect is wondering if the portfolio invested more in the sectors that performed better. Figure 11 shows the allocation effect for the whole portfolio.

Figure 16: Interaction against the random benchmarks during 2007 for Consumer Discretionary.

Figure 17: Interaction against the random benchmarks during 2007 for Health Care.

Figure 18: Interaction against the random benchmarks during 2007 for Materials.

Performance for decisions before 2007

Here we want to answer a second question.

Question 2: How good — during 2007 — were the decisions made prior to 2007?

The portfolio as it existed at the start of the year embodies those decisions.

To compare with this we want representatives from the realm of portfolios that could have resulted from such decisions. This is the set of portfolios that obey the constraints at the start of 2007.

That is, the random benchmarks we create are portfolios that — as of the end of 2006 — obey:

long-only

50-60 names

no asset with more than 4% weight

same value as the actual portfolio

The time period of interest is 2007, but the decisions of interest are prior to then. Hence all the portfolios will be static.

We can inspect plots to assess the quality of the decisions.

Figure 19 indicates that — overall — the decisions made prior to the start of 2007 were exceedingly mediocre during 2007.

Figure 19: Quality of decisions made prior to 2007 during 2007 — the “no trade” portfolio (green) versus the static benchmarks (gold). The selection, allocation and interaction plots reinforce the idea of mediocre results during 2007.

Performance for 2008

The final question that we’ll approach is:

Question 3: How good — during 2008 — were the decisions made in 2007?

Again in this setting the decisions are made prior to the evaluation time period. Hence the portfolios will be static. The pertinent portfolios are ones that we’ve already seen. The portfolio as it exists at the end of 2007 contains the fund manager decisions. The set of relevant benchmarks is the final state of each of the random benchmarks generated throughout 2007.

Figure 20 shows the performance — in returns — during 2008 for the 2007 trading decisions. Figure 21 is the corresponding picture of the performance percentiles through the year.

Figure 20: Cumulative returns during 2008 for the portfolio as of the end of 2007 (blue) and corresponding random benchmarks (gold).

Figure 21: Fraction of (static) random benchmarks better than the (static) portfolio during 2008. In the middle of the year the decisions were doing very well. But when the crash came, the portfolio decisions crashed even harder than the benchmarks.

Attribution for the whole portfolio

Figures 22 through 24 show the attribution for the 2007 decisions during 2008.

Details

survival

The universe is stocks that were in the S&P 500 in 2012, so there is survival bias in the universe. Since the same universe is used for all of the portfolios, there will not be survival bias in the current analysis. It is only a hypothetical portfolio anyway.

strategy

The strategy of the portfolio is to use the default signal from the MACD function in the TTR R package. So essentially a momentum strategy.

trading constraints

The alternative choices are done assuming the amount and timing of trading is exactly the same as what actually occurred. In actuality the fund manager would have had more latitude than that. Hence the distribution of the random benchmarks is likely to be (a little) too narrow.

An easy way to partially overcome this bias is to randomly choose the days on which the trading is to be done (separately for each benchmark). However, this is not a very attractive option if there is cashflow.

Another possibility is to force trading of at least the amount actually traded at each point, but allow some amount more — perhaps 10% or 20% more.

adjusting Brinson returns

The Brinson analysis (the one in the tables above) uses an adjustment to the sector returns because the aggregation of the returns didn’t match the portfolio return for the full year (the trading makes weights approximate). The simplest of adjustments was used: add the same value to each sector return that did not have a zero weight — see the R code below. The analyses represented in plots were not adjusted.

Summary

The technique presented here:

focuses on the decisions made in the portfolio

shows how good those decisions were relative to the alternatives available to the fund manager