After eighteen long lockdownerous months, it seems like benevolent Boris is finally going to set us free. So I thought now would be a good time to come clean about my new found secret passion. I luuuuurve data.
But when I look back over the various chapters of my life to date, I think I always have. I just never realised it.
A case in point, my favourite sports are cricket and basketball. If you allow either of those games to go on for long enough then you’ll be able to scrape together material for a decent highlight reel, but in their raw, unadulterated form, they are blindingly boring. They are however, two of the world’s statsiest sports. Those numbers are crack to me.
I also love football, but truth be told, my favourite outlet for that sport is old school Championship Manager – a game in which you pretend to have a job that requires you to endlessly pore over numbers out of 20.
If I could take all the hours I have spent on that game and re-invest them in music or languages instead, then I’m confident I’d be able to play trombone and write this blog in Japanese. Yes, both.
It’s for that reason that I can’t let myself get sucked into Fantasy Football or other forms of fantasy sport. I know how much I would love it, and how unproductively my time would be spent honing the skill. I just spent the last ten years shaking off online poker (another haven for the numerically aroused).
I realise most people don’t feel the way I do. They have only a passing interest in numbers and certainly don’t go to the effort of crunching them. Some can’t crunch, others can but don’t want to, and a few reluctantly crunch because their job demands it. But I crunch for fun.
Evolutionary psychology has an explanation for this situation. Simply, our ancestors didn’t need maths to survive. The ability to add and subtract in small numbers was occasionally useful, but no other operators were required for hunter gathering. They had no practical use for standard deviation.
That means our brains are not honed or structured for doing maths. Every numerical skill must be deliberately learned and practised, because almost none of it is built-in (unlike our fear of heights and snakes) and it does not come naturally (unlike eating or face recognition). It is testament to the amazing versatility of our brains that we can do any maths at all.
As I’ve previously discussed, anything that requires mental effort is hard, and we try to avoid it. We prefer mental shortcuts and estimations to precise answers because they’re quicker and usually good enough. Our default brain mode is “do it the easy way”, and beyond the basics, maths always requires some effort.
To illustrate our aversion and general lack of aptitude, I thought I’d discuss a couple of real-life data mistakes from my career in gambling. These involved people who were all intelligent enough to know better.
The Case of the Missing Conversion Cycles:
A common CRM program in the gambling industry is an automated cycle of communications (and promotions) for new customers who are yet to deposit. We called it the conversion cycle. They’re not hugely effective, but they are cheap to run and therefore usually profitable.
The trick with these programs is to innovate, test and iterate. Just keep tweaking and improving. However, for a reason known only to himself, the guy at the company who ran this cycle (his main responsibility) decided to turn it off, and just leave it off. Let’s call him Lazy Paul.
About a month past before someone from Regional Marketing asked for an update of conversion cycle performance. They were told that they had been turned off, and all hell broke loose. The Regional Marketing folks had been fishing for a scapegoat for recent underperformance, and now they had it. Blame was quickly attributed.
Unfortunately, the furious finger of fault was pointed too fast. The reason Paul hadn’t turned the cycles back on, is that his performance dashboard showed they were making absolutely no difference. This was not the heaven-sent excuse the regional folks were praying for and the whole marketing storm in a teacup was swept under the marketing carpet of shame.
There’s a lot of incompetence in that short tale, but the real crime is that it stopped there. Everyone got a bit embarrassed and moved on before any useful answers were obtained.
Lazy Paul’s findings were only indicative that the conversion cycle was ineffective. There was no clinical A/B test, no control group, no statistical rigour whatsoever. All we knew is that the pre-stoppage numbers looked the same as the post-stoppage numbers, and that is far from conclusive. It’s grounds to dig deeper, not to give up.
The real reason why the numbers were looking bad, is that it was early summer. People are less interested in using computers when the weather is getting nicer. Every year management would forget that simple fact and hit the panic button.
Not long after that, Paul moved onto new projects, then quietly left the company. Responsibility for these automated cycles fell under my remit, and split testing became standard practice.
Edit: That isn’t supposed to be a boast, split testing is the bare minimum required for getting reliable answers. Any idiot would have made that change, the idiot just happened to be me.
Worshipping a Rubbish Key Performance Indicator
When I took over running the PokerStars.net Play Money product, I got a handover pep talk from the guy I was replacing. Let’s call this one Smug Martin. “Freemium gaming,” Martin insisted earnestly “is all about ARPDAU. That is the only metric that matters.”
In normal people words, Average Revenue per Daily Active User (ARPDAU), is how much each of your current customers are spending per day. Making business decisions based solely on the metric is akin to picking a basketball team using exclusively BMI data. You can keep fat lads off the court, but you’re almost certainly going to lose.
But Martin wasn’t alone. His new boss and the top dogs at the company were all on the ARPDAU train. They must have read about it somewhere and decided it sounded clever enough to be useful.
Behavioural economics tells us that people are generally sensitive to their incentives – so they will take decisions that lead to the best financial outcomes for themselves. Based on Martin’s ARPDAU advice, my first decision would be to cut off the 50% least profitable players.
As it happens, at least 90% never make a purchase, so I’ve theoretically lost out on nothing and doubled my ARPDAU. But why stop there? I may as well get rid of anyone that is purchasing below the current average.
With the way purchases are skewed in freemium gaming (most of revenue comes from a fraction of customers known as ‘whales’), that move takes me down to perhaps 2% of my original traffic.
My ARPDAU is now through the roof! I was making $1 per active customer per day, but now I’m making $40. I’ve also alienated 98% of my customers and dropped 20% of my revenue. Based on the incumbent expertise, those millions of lost dollars would be considered incredible success.
I never did pay any attention to ARPDAU, but my superiors did despite my protestations. Thankfully, steady natural attrition meant my denominator kept getting smaller, and my CRM efforts coaxed a little more out of those that stuck around, so it always looked good.
The moral of the story is that numbers are hard, but really important. If you can’t understand them, or don’t want to, find someone you trust to do it for you. I’m available and if your data is any good, I’ll look at it for free. I need a fix.
Bobby’s Bets
Germany and Switzerland have crashed out, so my Calcutta losses are pretty ugly. Spain have also lost so that’s another £7.50 gone. Importantly though England are hanging in there, so my £10 sweepstake entry is live. If that comes in I’ll probably finish up ahead out of pure blind luck.
Reluctant Gambler 