← Back to Episodes
Why do holdout tests give unreliable results?
Control groups get contaminated and skew results.
S: At plus eighteen percent and trending up week over week, the reported test lift is strong.
C: The contamination mechanism operates through three pathways simultaneously.
I: When holdout group contamination inflates the reported lift, budget scaling decisions are made against a distorted signal.