Imagine starting a new job and having to drive to the office on your first day. You’ve never been there before, but you know it’s next to a gym you used to go to. From experience, you remember that it typically takes about 30 minutes to get to the gym at that time of day — even though the last time you went was over a year ago (you eventually stopped going, but that’s another story).
Now it’s your first day on the job, and you need to decide what time to leave the house. Let’s imagine you don’t have access to GPS or navigation data. What would you do?
- Use the information you already have (even if it’s outdated) and plan for a 30-minute drive?
- Or ignore everything you know and randomly pick a departure time?
Naturally, the first option makes more sense — you’d rely on the information you have rather than ignore it completely.
Well, if that reasoning makes sense to you, then you’re already on your way to understanding a Bayesian approach to statistics.
Who was Bayes?
The name “Bayes” has likely intimidated every statistics student at some point, referring to the 18th-century mathematician known for a complex theorem about probability — one that has haunted many pre-exam nights. He is the austere face depicted in the image of this article. But in reality, the Bayesian approach to hypothesis testing is more intuitive than it seems.
It simply involves considering a prior assumption — before analyzing data or starting an experiment — and then updating that assumption as new data is gathered.
How does it work? A practical example
Take, for example, a marketing conversion test: you want to measure what percentage of people exposed to an ad actually buy the product.
If a similar test was conducted in the past and showed a 5% conversion rate (i.e. 5 out of 100 people exposed made a purchase), then 5% becomes your prior assumption for the new test.
As fresh data is collected, you’ll start forming a new idea of what that conversion rate might be — this is called the likelihood or evidence — and the final result of your test will be a combination of both the prior and the likelihood.
Naturally, the weight of each will vary depending on how much data you collect. The larger your dataset, the more influence the evidence will have. If you only have limited data, the prior (i.e. the result from the previous test) will play a bigger role in the final result.
Back to the original example: during your first week of commuting, the data you gather on travel time might be limited and highly variable — statistically speaking, it would have a high variance — so your prior estimate (30 minutes) would still significantly influence your final estimate.
But after a few weeks, the data starts to stabilize around a new average — say, 20 minutes — with less variation. At that point, it makes sense to update your estimate based on the actual evidence you’ve gathered.
The rivalry: Bayesians vs. Frequentists
It all sounds logical — but in the world of marketing testing, the Bayesian approach is still far less common than the more traditional Frequentist approach, which relies solely on observed data. The Frequentist method disregards prior knowledge and bases all conclusions on the likelihood alone.
This works well with large datasets, but as you might guess, issues arise when — due to time, budget, or other constraints — you can’t collect a large enough sample. In those cases, the Frequentist results can be inconclusive.
But there’s another valuable aspect of the Bayesian approach: it allows for probabilistic conclusions, which the Frequentist method does not.
Back to our marketing example — say you’re comparing two different ad campaigns. The Bayesian method lets you conclude something like: “Tactic A has an 80% probability of being more effective than Tactic B.”
That’s because instead of estimating a single value (like average conversion rate), the Bayesian method estimates a distribution of possible conversion rates for each tactic. You can then compare those distributions and draw probabilistic conclusions — a powerful advantage for decision-makers.
Perhaps the most important benefit from a business perspective is that the Bayesian approach converges on a reliable result much faster than the Frequentist one.
To reach a 95% confidence level that one version is better than another, the Bayesian method typically requires a much smaller sample size.
Take a look at the chart: each bubble shows the minimum required sample size for various combinations of test assumptions. The orange bubble represents the Bayesian minimum size, while the blue bubble shows the Frequentist one. As you can see, the orange bubbles are nearly always smaller — on average, about 50% smaller — although it varies depending on the initial assumptions.
So why choose the Bayesian Approach?
To sum up, the Bayesian method offers three major advantages:
- Smaller required sample sizes
- Ability to incorporate prior knowledge
- Probabilistic (not just binary) conclusions
So don’t let the intimidating name or the perceived complexity scare you off. The Bayesian approach is intuitive, logical, and incredibly practical — especially when data is limited and decisions must be made under uncertainty.
So… why not give it a try?