Tag Archives: a/b test

Let me set the scene: You’ve just attended a conference and your head is buzzing with new ideas. But there’s one in particular that you have to try! You know your organization’s donors will love it and really respond to it. So you go to your decision-maker and put the idea in front of her. She immediately shuts it down. “Our donors won’t respond to that!”, she says. You, deflated, go back to doing the same old, same old.

Here’s another scene: You just got an email from your favourite charity (but not the one you work at). It has a lot of copy, yet you’re captivated the whole time you read it. You get to the end and can’t help but make a gift. But there was so much copy! That’s not best practice! And yet… you gave. So you start thinking about the charity where you work. You only write short copy emails there; it’s “best practice”, after all. You consider making your next email a bit longer, but the digital experts in your office might say no. And what if it sacrifices revenue?

If you’re like most fundraisers, one or both of these situations is familiar to you. Revenue is precious, and if it’s coming in, then there’s not a revenue problem. If there’s no problem, there’s nothing to fix. If there’s nothing to fix, why try something new?! New ideas have no place in your charity.

But there is a problem! Status quo is a problem. Maybe not today, maybe not tomorrow, but at some point things will plateau if they haven’t already.

And chances are, the most senior people at your charity don’t want to see revenue go flat. They want to see it go up.

But how?

We use a quote from Albert Einstein a lot in our office to shake us out of status quo:

So how do you stop being insane? How do you challenge your known best practice without sacrificing (too much) revenue?

You test.

In my opinion, we don’t do nearly enough testing in fundraising. We’re so risk-averse – and understandably so. We don’t have money to just play around with, and for most charities it would be irresponsible to “play” with our money anyway.

But there’s so much to learn by trying new things!

Testing is the way. Testing is the way to convince your organization’s decision-maker to try something new. You go status quo with one group of donors – maybe send them the year-end package you always have. And then with another group of donors – send them the new package! Or the long-copy email you’ve been wanting to try!

6 Tips for Testing:

Test ONE thing. To have a true test, you have to be able to identify the one thing that made one “package” more successful than the other.

You don’t have to do a 50/50 test, you can do 75/25 or whatever you want! But – for the test to be conclusive, you want to have a minimum of 100 responses.

Don’t just test for testing’s sake. If the element that you’re testing “wins”, the assumption is that you’ll roll out that element in future efforts, so it needs to have value.

Digital testing is the same as testing through mail. You’re still testing one thing.

You can test more than just creative/copy elements. You can also test data elements. What would happen if you mailed 50% of your inactive donors a totally different letter than other donors? Would they perform better? Or what if you tested your usual ask grid ($50 $100 $250) against something a bit more aggressive? Would that result in a higher average gift?

Sometimes – even without testing it – it’s time for your longstanding control package to go. (Note: Your “control” is the package that keeps on winning against test packages.) Just because it’s “winning” doesn’t mean it should still go out in the mail. Sometimes it’s time for something new, with or without testing.