Symptom
At plus eighteen percent and trending up week over week, the reported test lift is strong.
You asked
A holdout group has been set up alongside your active campaigns, and early results show a measurable difference between exposed and non-exposed users. The test appears to be running correctly, with clear reporting on performance lift between the two groups.
Symptom
At plus eighteen percent and trending up week over week, the reported test lift is strong.
Cause
The contamination mechanism operates through three pathways simultaneously.
Impact
When holdout group contamination inflates the reported lift, budget scaling decisions are made against a distorted signal.
A holdout group has been set up alongside your active campaigns, and early results show a measurable difference between exposed and non-exposed users. The test appears to be running correctly, with clear reporting on performance lift between the two groups.
At plus eighteen percent and trending up week over week, the reported test lift is strong.
The contamination mechanism operates through three pathways simultaneously.
When holdout group contamination inflates the reported lift, budget scaling decisions are made against a distorted signal.