The “So What?” Doctrine

My thinking started a few years ago when I was working with a large disability charity on a multi-channel campaign. The charity’s influencer agency had promised shares and retweets from famous comedians and A-list rock stars with millions of followers. Great news — no one can argue with this kind of “exposure” - and the A-List rock star was one of my childhood heroes. I was impressed.

However, it quickly became clear to me that the agency were pinning their colours to metrics which didn’t prove success. Did anyone click the retweeted link? What % donated to the campaign after seeing the tweet? Which influencer’s retweet had the biggest impact? Perhaps unsurprisingly, my line of questioning (and suggestions around granular UTM tagging) didn’t go down too well. A retweet to a million followers sounds too good to challenge (we welcome Twitter’s recent announcement that they’re rolling out a prompt when a user on the platform retweets a link they haven’t opened).

The So What? Doctrine is a set of measurement principles against which a nonprofit’s digital marketing should be built. It may mean rewriting your current reporting methodologies, but it’ll give you real insight into the success of your campaigns. It underpins every campaign we work on, and forms the basis of the training we give our team.

As a rule charities don’t push themselves hard enough when measuring the success of digital marketing campaigns. Charities (and their agencies) routinely use metrics that only tell part of the story and have no relation to wider organisational objectives and KPIs (“we got this from the API!”). This style of reporting is like throwing dust in a nonprofit’s eyes and running away. It’s the most common style of reporting.

Furthermore, the real results are often obscured in reporting. It’s fine that your Christmas campaign raised £50K in revenue, but how much did you spend to raise it? It’s fine that you increased revenue by 30% year-on-year, but from what starting point, and with how much additional media spend? [1]

The So What? Doctrine requires nonprofits to engage with and rethink how they report, in two ways.

Firstly, ask why you measure. If you don’t know why you’re not sure your metrics are giving you a clear picture, now’s the time to stop, and reassess the basics. To do this, develop a digital marketing measurement framework that:

• Adapts your organisational objectives, only those that marketing can influence. If your charity has published an annual report or “5-year vision”, this is where you should start.
• Maps specific goals to these objectives.
• Lists out, channel by channel, the metrics (KPIs) that indicate whether that goal has been achieved.

Include two columns: “What does this metric tell us?” and “How can we influence this metric via digital marketing”. If you can’t provide robust answers for both of these questions, replace the metric with something more robust.

This way every metric sits under a corresponding goal, which in turn sits under an objective. It’s fine to track follower numbers on a social media platform, so long as you’re clear why you’re tracking those numbers. Building the model takes time, and buy-in from everyone you expect to use it.

This framework establishes a series of primary objectives as well as contextual metrics. If you’re running a campaign to deliver newsletter signups, then newsletter signups is your primary KPI, but you also need contextual metrics in your reporting to understand the whys and hows of a campaign. In the example I’ve given, these metrics might include clicks and conversion rate (to indicate the size of the campaign, and whether our targeting was good). These metrics will provide you with two things: an insight into how a campaign performed relative to other campaigns; and an explanation as to why a campaign succeeded or failed. Comparing two YouTube campaigns designed to deliver against a primary KPI of donations is useless if you don’t also measure a contextual metric like “no. of views”.

The second area of focus in The So What? Doctrine is what you classify as “success” A common trap nonprofits fall into is to settle for the metrics that are easily available. Get out of this comfort zone; develop a healthy mistrust of “default metrics”. Employ a data analyst to measure better things, and raise the threshold against which you judge success.

Begin your journey by ensuring you understand exactly how each digital marketing platform’s metrics are calculated. Facebook’s Video views is a good success metric only if success means a user watched the first three seconds of your video. Reach tells you nothing about the specific success of a post or campaign, just that a platform’s algorithm is broadly happy with past success. Post engagement tells you four things, not one (reactions, comments, shares, clicks—all of which indicate something different about your success rate). I won’t beat up on Bounce Rate, a metric last seen holed up at the Hotel of Most Abused Metrics Ever, I’ll just assume you place no great weight on it.

If you’re promoting a piece of content, and success is people reading that content, ditch “bounce rate” (I know, I promised!). Ensure you have tracking in place to track not just “scroll depth” (the noisiest of all the metrics) but “scroll depth PLUS time on page”. This can be tracked with tools like Google Tag Manager. Try placing a monetary value on specific actions—£1 for “75% content read”, £2 for a PDF download, £3 for 80% of a video viewed. While it’s difficult to be scientific about these values, this will at the very least help you understand relative ROI.

If you’re running an individual giving campaign, ensure e-commerce tracking is set up well. Split out Gift Aid, so you can report on it independently. Ensure you’ve thought through how to report on Regular Giving donations to factor in internal understanding of attrition and retention rates (Google Analytics doesn’t report repeat transactions in an intuitive way). Bring Assisted Conversions into the mix as a standard reporting feature—Google Analytics’ default measurement model tells a crude picture when it comes to attributing conversions and revenue. Understand how the various conversion windows work so you don’t panic when your Display campaigns don’t deliver any revenue right off the bat.

Measure, then measure better.

Ultimately, be honest. Does that metric really indicate success when tied to a clear organisational objective? What should you really have measured on your last campaign, and what does it really tell you? Here’s the sticker I keep on my desktop that guides the measurement work we do at Torchbox.

Feel free to copy it.

[1] The difference between return on advertising spend (ROAS) and return on investment (ROI) often isn’t understood or explained properly. ROAS doesn’t factor in the £1 you spent on advertising, so it doesn’t tell you a true picture. It always looks better than ROI, which does factor in that £1. Similarly, a statement like “this campaign’s activity had an ROI of 900%” is extremely difficult to understand or contextualise. That’s why we always frame our ROI metrics to charities as “For every £1 you spent, you generated £10 of profit”.

[2] We use Avinash Kaushik’s model as our starting point.