Your platform's ROI number is grading its own homework
Most marketing reports credit a campaign with every order its recipients placed afterwards. Many of those orders were coming anyway. The fix is old, cheap and almost nobody does it.
Open almost any marketing tool used by restaurants and you will find a big, cheerful number near the top of the campaign report. Revenue generated. Return on spend. Sometimes it says twenty times or thirty times. It is meant to make you feel good, and it usually does.
Here is how that number is normally worked out. The campaign went to a list of customers. Some of those customers ordered in the following few days. Their orders are added up and credited to the campaign.
Spot the problem. Some of those customers order from you every week. They would have ordered that week whether or not the email arrived. The report gives the campaign credit for them anyway.
Attributed is not the same as caused
There are two different questions hiding in that report. The first is how much the people who got the message spent afterwards. That is attributed revenue, and it is easy to measure. The second is how much more they spent because they got the message. That is incremental revenue, and it is the only number that tells you whether the campaign was worth doing.
The gap between the two can be enormous, especially for messages sent to your best customers. Your regulars have high attributed revenue on every single campaign, because they were always going to order.
Hold some people back
The way to find the true number is a holdout. Before the campaign goes out, take a random slice of the eligible customers, say one in ten, and do not send them anything. Then compare the two groups over the following weeks.
If the people who got the message ordered more than the people who did not, the difference is what the campaign actually did. If the two groups look the same, the campaign did nothing, no matter what the cheerful number at the top of the report says.
It feels wasteful to leave customers out. It is not. A holdout is the cheapest research you will ever do, and it stops you spending real money on campaigns that only look good.
What to do with the answer
Once you start holding people back, a few patterns usually show up.
- Discounts sent to frequent customers often show a small true lift and a large cost.
- Messages to customers who have started to drift away often show the biggest real effect.
- Some campaigns that look average on the attributed report turn out to be the best things you run.
Keep the holdout small, keep it random, and run it on every regular campaign rather than once as a special project. After a few months you will know which parts of your marketing are pulling weight. You will also trust your reports again, which is worth more than any dashboard.
