A 2013 result became one of the most repeated findings in the humanities: read a few pages of and your ability to read other minds improves on the spot. A later team failed to replicate it, the original authors objected on three specific methodological grounds, and this paper is the answer to those objections. Including a reanalysis done the way the critics asked for. What came out of that?
The reanalysis failed too. Kidd and Castano had argued the replication wrongly kept people who had not read the texts, mishandled , and wrongly kept people who skipped the . When they reanalyzed the replication data in the manner they themselves argued was correct, the original pattern still did not appear. So the replication team reaffirmed finding no evidence that literary fiction uniquely and immediately improves .
The 2013 claim was unusually attractive and unusually specific: reading a short passage of literary fiction improves theory of mind immediately, and popular fiction and nonfiction do not. It traveled a long way, into reading campaigns and newspaper columns, on the strength of a brief laboratory effect.
A replication attempt by Panero and colleagues found nothing. Kidd and Castano then critiqued that attempt on three grounds. It failed to exclude people who had not read the texts, it suffered a failure of random assignment, and it failed to exclude people who had not taken the Author Recognition Test. This paper is the point-by-point reply.
The decisive move is not rhetorical. When Kidd and Castano reanalyzed the replication data in the way they argued was most appropriate, they still failed to reproduce the pattern of results from their own original study. The objections were answered by granting them and finding the effect absent anyway.
So the replication team reaffirm their conclusion: no evidence that literary fiction uniquely and immediately improves theory of mind. They are careful about what that does and does not mean. Their stated objective is not to prove that reading literary fiction fails to benefit social cognition. It is to call for deeper research into how any such effect could be measured, and to temper the claims made meanwhile. A null result in a noisy paradigm licenses caution rather than a counter-claim.
wat. Even reanalyzed exactly as its defenders demanded, the data refused to show a short story sharpening anyone’s read on other people.
wat wat
This is what a dispute looks like when it is settled on the data rather than by repetition. A critic reanalyzing someone else’s data to their own specification, and still not finding the effect, is about as clean as this gets.
How do we know?
Panero and colleagues answered each of the three critiques of their failed replication, and reported that the critics’ own preferred reanalysis of that data still did not reproduce the 2013 pattern.
What is still unknown
A failed replication is not a disproof. The authors say explicitly that their aim is not to show literary fiction has no social benefit, but to call for better measurement and more tempered claims.
Learn something?
Where to next
Can a few minutes with a text change how well you read another person?literature
Does a spoiler really make a story more enjoyable?literature
Does reading literary fiction make us more empathic?literature