GA4 vs your gut: when to override the data
We built our practice on data discipline. Sometimes the data is wrong, and the right call is to ignore it. Here's how we tell the difference.
If you've been on this site for five minutes, you know we lean hard on data. 'Data over opinion' is one of our four written principles. So it might surprise you that the most senior people on our team will, occasionally, make calls that the data tells them not to make.
Here's how we think about it.
When to trust GA4 absolutely
- Comparisons within the same time window: this campaign vs that campaign, this week vs last week
- Direction of travel: is this metric trending up, down, or flat
- Your own pre-defined success threshold: did the test hit the goal you set before running it
When to be skeptical
- Cross-channel attribution after a big iOS update — Meta probably under-credited
- Sample sizes under 100 — variance is doing more work than the signal
- Comparing two time windows that span a holiday or major external event
- Anything where the data contradicts a strong qualitative signal from sales or support
When to override entirely
Sometimes the data says one thing and the founders say another. The right call is usually to weight the founder signal heavily — they're closer to customers than the dashboard is. Then design a small test to validate or kill the founder hypothesis, and let the next month's data resolve it.
The discipline is to override the data on purpose, not by accident. If you're overriding the dashboard every week, you don't have a data culture — you have a dashboard you don't trust.