A famous 2009 study reported that people who run several media at once filter distractions worse, and the claim felt too true to question. Later studies did not all find it. Two replications then ran 14 fresh tests and reported every one of them, and a meta-analysis pooled the literature. What survived?
Very little. Of the 14 tests in the replications, only five showed the effect at all, and a stricter Bayesian analysis thinned those further. Pooling 39 effect sizes from the published work gave a weak association, and after correcting for the tendency of small studies to report large effects, the weak association was no longer significant.
The original finding is the kind that spreads fast because everybody already believes it. Ophir, Nass and Wagner reported in 2009 that went with worse , and the result became common knowledge.
Subsequent studies did not all reproduce it, which is where this work starts. Two new replication studies produced 14 tests with an average of about 0.81, or four chances in five of catching a real effect. The authors then reported all 14 rather than only the ones that worked.
Five of the 14 came out statistically significant in the expected direction, and a conservative Bayesian analysis cut into those further. Reporting every test is what makes that visible; publishing only the successes would have looked like confirmation.
Then the meta-analysis of 39 effect sizes from the literature found a weak association, which became nonsignificant once were corrected for. The authors go as far as questioning whether the association exists in laboratory tasks at all. Both measures here are narrow, a self-report questionnaire and a brief laboratory task, so what falls is a specific claim rather than every worry about screens.
wat. Across 39 tests of multitasking and distraction, a weak link disappeared once small-study effects were corrected for.
wat wat
A habit that feels scattered is not by itself evidence of a lasting change in the ability to pay attention.
How do we know?
Wiradhany and Nieuwenstein ran two replication studies with an average statistical power of 0.81, and combined the published tests of distractor filtering against media multitasking into a meta-analysis of 39 effect sizes.
What is still unknown
A self-report questionnaire stands in for media use and a short laboratory task stands in for attention, so this closes one specific laboratory finding rather than every effect screens might have on a life.
Learn something?
Where to next
Why is it so hard to say what color a word is printed in?attention
Would you notice if the person you were talking to was swapped for someone else?attention
How can anyone miss a gorilla walking through the middle of the screen?attention