- At Prefect we market to developers, notoriously very privacy-conscious. Even within this population only about 10% use ad-blockers, so it's lower than you think.
- You could do this, or you could only look at the populations that don't have ad-blockers, with an assumption that both test groups are evenly split (if properly randomized) across this population too. It's not perfect, but depends on how quick and broad strokes you want to get the results.
Hey Sara,
thanks for writing this post!
In case a significant number of users use ad-blockers would you use instrumental variables to analyze the experiment?
Good question, a few thoughts:
- At Prefect we market to developers, notoriously very privacy-conscious. Even within this population only about 10% use ad-blockers, so it's lower than you think.
- You could do this, or you could only look at the populations that don't have ad-blockers, with an assumption that both test groups are evenly split (if properly randomized) across this population too. It's not perfect, but depends on how quick and broad strokes you want to get the results.