Apparently ‘nudging’, one of the cornerstone findings of pop psychology that has borne many of books and TED talks, may be beset by publication bias:
The death knell for nudging?
“When this publication bias is appropriately corrected for, no evidence for the effectiveness of nudges remain.”https://t.co/LzHNvG7h6m pic.twitter.com/LVVXYSupVL
— John B. Holbein (@JohnHolbein1) July 24, 2022
Publication bias means that studies that confirm a certain hypothesis are more likely to be published compared to studies which disprove it or are inconclusive.
Despite similar quality of execution and design,[2] papers with statistically significant results are three times more likely to be published than those with null results.[3] This unduly motivates researchers to manipulate their practices to ensure statistically significant results, such as by data dredging.[4]
From the paper No evidence for nudging after adjusting for publication bias:
Thaler and Sunstein’s “nudge” (1) has spawned a revolution in behavioral science research. Despite its popularity, the “nudge approach” has been criticized for having a “limited evidence base” (e.g., ref. 2). Mertens et al. (3) seek to address that limitation with a timely and comprehensive metaanalysis. Mertens et al.’s headline finding is that “choice architecture [nudging] is an effective and widely applicable behavior change tool” (p. 8). We propose their finding of “moderate publication bias” (p. 1) is the real headline; when this publication bias is appropriately corrected for, no evidence for the effectiveness of nudges remains (Fig. 1).
We conclude that the “nudge” literature analyzed in ref. 3 is characterized by severe publication bias. Contrary to Mertens et al. (3), our Bayesian analysis indicates that, after correcting for this bias, no evidence remains that nudges are effective as tools for behaviour change.