And yet, everyone wants to talk about investment performance. It’s all over the media. It’s what almost everyone asks about. But investment performance isn’t enough.

Investment costs, when not accounted for, will obliterate investment performance.

There’s a second and related truth to that. Investment performance rises and falls. No one earns market-defying gains year after year. But investment costs – those just keep on coming. While some investment costs are tied to performance, many are not. As you’re about to see, you should stop asking so many questions about performance, and instead ask more questions about costs.

Especially if you have a wealth manager or financial advisor who says he charges no fees. He makes money somehow!

Let’s explore how investment costs can devalue what appear to be strong performance gains.

Performance-Based Fees

Suppose your advisor gets a bonus fee if your investment performance achieves certain milestones. That cost hurts you in the long term. Why? Because it’s unlikely your portfolio will continue performing at that level year after year.

So, let’s say one year your portfolio earns 22% interest, and this triggers a one-time bonus for your advisor. Let’s say that bonus is $10,000.

Now, if you have several million dollars in your portfolio, you might think his strong performance merits a performance-based fee like this. It’s motivating for him. You still made a lot of money, right?

But what happens next year? What if he only gets you 8% next year? Or 4%. Suppose your average gains in the 20 years after your 22% banner year is a modest but respectable 6%.

How much would $10,000 be worth after 20 years at just 6% growth?

$32,071.

It may seem ‘fair’ to reward your financial advisor for a good year. But when viewing your portfolio over the long term (the only term that matters – who cares about one isolated year?), all that bonus did was cost you $32,000.

Bond Spread Investment Costs

When you buy and later sell a bond, the difference between the sell price and the purchase price is called the spread.

As an analogy, picture a used car dealer who buys a car from someone, and then sells it at the higher retail price. The difference in these two prices is like the spread, and the dealer pockets the difference.

Likewise, when a bond manager sells bonds, the manager can take a percentage of that spread as payment. (Hidden cost alert: This is one way the managers who charge “zero fees” make their money).

Suppose your bond manager buys some bonds as part of your portfolio for $150,000. They increase in value. Some time later, he sells them for $250,000. Sounds great right? But if your manager is charging a percentage of the spread, you don’t get all of that $100,000 increase.

If your bond manager charges a 5% fee on the spread, that’s $5000 you’ll never see.

How much would $5000 be worth after 20 years at just 6% growth?

$16,035.

Margin Interest Costs

Some financial advisors get paid depending on how much money they’re managing. A non-fiduciary advisor therefore looks for opportunities to get more money under her control.

One way they do this is by encouraging their clients to borrow against their assets to free up some liquidity. This is kind of like a home equity line of credit, but with your investments instead. The money you borrow goes on margin. That act triggers two new costs for you.

First, you’re now paying interest on it, on money you already had. Second, now that your advisor is overseeing more money, his fees increase.

Suppose you borrow $100,000 against your assets at 4% interest, and your advisor gets a 1% fee. Every year after that, until you pay it back, you’re paying an additional $1000 to your advisor and $4000 in interest.

Let’s say you do that for five years, and then decide to pay off the debt. In that five years, you spent approximately $25,000.*

How much would $25,000 be worth after 20 years at just 6% growth?

$80,178.

That’s only $20k short of what you borrowed against to begin with. In effect, your borrowed it twice, but only got to use it once.

*This computation would in reality be more complicated than this. We’ve simplified it so you have a fair sense of how much these ‘small’ costs can add up.

Internal Bank Investment Costs

Many investors – especially high net worth individuals – who turn to big banks for the perceived “simplicity” end up paying dearly for it.

Suppose you invest $500,000 with one of the big banks. They tell you they’ll diversify your investment so it’s allocated in a healthy mix of equities, target-date funds, bond funds, and various other categories. Sounds good right? Asset allocation and diversification are important, so it seems like they’re taking a lot of work off your back.

The thing is, unless you ask, you don’t know what internal fees that big bank will be charging for all these investments. Often, they will use funds managed by their own bank, and these might have much higher fees than your other options.

Suppose $250,000 gets put in a target date fund with a 1.25% management fee. $200,000 goes in an equity fund with a 1.5% fee, and $50,000 goes in a bond fund with a 1% fee.

The value of these investments will grow each year so it’s impossible to get precise numbers for calculation. But let’s make some conservative estimates:

Target-date fund fee: $3125/yr and increasing from there.

Equity fund fee: $3000/yr and increasing from there.

Bond fund fee: $500/yr and increasing from there.

Let’s say you keep this arrangement for five years. Again, roughly estimating conservatively, this comes to $35,000 in fees over five years. ($6625 x 5, then rounding up a bit).

How much would $35,000 be worth after 20 years at just 6% growth?

$112,250.

How much lower could this number be? Suppose you invested all this money in passively managed funds charging 0.2% fees instead of the big bank’s high-priced actively managed ones.

$500,000 x 0.2% = $1000/yr and increasing from there.

Estimating five years plus some increase, let’s say the five year fee comes to $5500.

How much would $5500 be worth after 20 years at just 6% growth?

$17,639.

So with the higher priced internal fees from active management in the big bank’s funds, you lost almost $100,000 in growth over 20 years. And this assumes the bank’s funds perform at about the same level as the passively managed portfolio, which is unlikely because actively managed funds almost always do worse.

In fact, let’s talk about that one next.

Active vs Passive Management

Active managers bet on the market with your money. That’s the nature of active management. They’re guessing. And you’re hoping they guess right more often than not. Active managers spend their time buying and selling, trying to time the market, capitalizing on ‘hot’ buys and new trends.

Because of all the work and time that go into this, active managers charge higher fees than passive ones.

Suppose you invest $2 million with an active money manager, and he charges a fairly standard fee of 1%. (though it can be much higher).

In just one year, your 1% fee amounts to $20,000.

You’ll pay that every year, and it will increase as the value of your portfolio increases.

A passive manager, by contrast, might charge as little as 0.1%. To make it a bit more fair, let’s double it to 0.2%. At that rate, the first year’s fee on $2 million will be $4000.

The difference between these is $16,000.

How much will $16,000 be worth after 20 years at just 6% growth?

$51,314.

And remember – that’s how much 20-year growth you’re missing out on from just a single year of active vs passive management fees. If you keep betting on the active manager year after year, you’re losing literally hundreds of thousands of dollars in lifetime value, soon to be millions if you keep it up.

If you’re an ultra high net worth individual, all these examples should be rerun with numbers in the $10 million, $20 million, and $100 million dollar range. Now, “costs” doesn’t feel like an adequate word for what you’ll be losing, especially if you combine all the various investment costs we’ve discussed so far. “Travesty” feels more appropriate.

These events get taxed at the marginal tax rate, currently 37%. But if you confine your investments to ones that only earn long term capital gains, your gains will be taxed at 20%.

37% tax, or 20% tax. Which do you prefer?

The passive approach produces less disruption, fewer changes, and this lower tax rate.

Suppose you invest $1 million through a financial advisor who uses active management principles. If you earn 10% growth, that amounts to $100,000 the first year. But because this manager’s active approach utilized short-term capital gains, you’ll pay 37% tax on those gains (in addition to his higher management fee!).

37% of $100,000 in gains is $37,000 in taxes that go to the government.

In contrast, had you earned the same 10% using passive management, and thus earned only long term capital gains, now you’ll be taxed at 20%, which amounts to $20,000 in this case. That’s a difference of $17,000. And again, this is only one year. This happens year after year after year.

How much will $17,000 be worth after 20 years at just 6% growth?

$54,521.

High Investment Costs Rob Your Long Term Growth and Performance

Now, as a high net worth person, you could very conceivably be mired in all these performance swamps at the same time – with the numbers used as reasonable examples. And to be clear – this is NOT an exhaustive list. We didn’t even talk about additional investment costs like commissions and tax loss cultivation. The numbers are just as big for those.

But assuming you are paying just the six investment costs we’ve detailed, you would have paid:

Performance-based fees for one good year

Bond spread fees you never knew about

Margin costs on borrowed assets

Internal big bank costs

Active management fees

Short term capital gains taxed at the marginal 37% rate

If you used all the numbers in these examples at once, added up all the investment costs and calculated how much 20-year growth you would be missing out on, it would add up to $346,369 in lost wealth.

How great would your wealth advisor need to be to overcome all that avoidable loss?

A far more sensible approach would be to simply avoid these investment choices.

Reduce your taxes.

Cut out the active managers.

Stay away from big banks.

Refuse to work with any wealth managers who aren’t fiduciaries.

Slash your costs, and you relieve the pressure to earn such high and often unsustainable rates of return year after year. Don’t worry about investment performance until you’ve minimized your investment costs. That’s where real money gets lost.