{"id":387,"date":"2026-04-23T10:52:42","date_gmt":"2026-04-23T14:52:42","guid":{"rendered":"https:\/\/psychexamples.com\/wordpress\/?p=387"},"modified":"2026-04-23T10:52:42","modified_gmt":"2026-04-23T14:52:42","slug":"ignore-effect-size-at-your-peril","status":"publish","type":"post","link":"https:\/\/psychexamples.com\/wordpress\/2026\/04\/23\/ignore-effect-size-at-your-peril\/","title":{"rendered":"Ignore Effect Size at Your Peril"},"content":{"rendered":"\n

This article<\/a> in The Economist<\/em> entitled \u201cRemoving Phones from Classrooms Improves Academic Performance\u201d (https:\/\/papers.ssrn.com\/sol3\/papers.cfm?abstract_id=5370727) has been cited by Chris Ferguson as an example of how ignoring effect sizes can lead to false conclusions. Here is the abstract of the paper, followed by Ferguson\u2019s critique:<\/p>\n\n\n\n

    <\/ol>\n\n\n\n

    Widespread smartphone bans are being implemented in classrooms worldwide, yet their causal effects on student outcomes remain unclear. In a randomized controlled trial involving nearly 17,000 students, we find that mandatory in-class phone collection led to higher grades \u2014 particularly among lower-performing, first-year, and non-STEM students \u2014 with an average increase of 0.086 standard deviations. Importantly, students exposed to the ban were substantially more supportive of phone-use restrictions, perceiving greater benefits from these policies and displaying reduced preferences for unrestricted access. This enhanced student receptivity to restrictive digital policies may create a self-reinforcing cycle, where positive firsthand experiences strengthen support for continued implementation. Despite a mild rise in reported fear of missing out, there were no significant changes in overall student well-being, academic motivation, digital usage, or experiences of online harassment. Random classroom spot checks revealed fewer instances of student chatter and disruptive behaviors, along with reduced phone usage and increased engagement among teachers in phone-ban classrooms, suggesting a classroom environment more conducive to learning. Spot checks also revealed that students appear more distracted, possibly due to withdrawal from habitual phone checking, yet, students did not report being more distracted. These results suggest that in-class phone bans represent a low-cost, effective policy to modestly improve academic outcomes, especially for vulnerable student groups, while enhancing student receptivity to digital policy interventions.<\/p>\n\n\n\n

    Ferguson says: This is a great example of why study authors relying on p-values and ignoring the likelihood of weak effect sizes being statistical noise can do great harm. The study looked at 17000 students in a study of cellphone bans in schools…still found null effects for most outcomes, but focus on one positive outcome with a d = .086 (about r = .043). Aside from the likelihood the one finding (against the other null findings) is due to chance, this effect size is well within the range of those indistinguishable from statistical noise (Ferguson & Heene, 2021). It should never have been interpreted as hypothesis supportive. Yet the Economist (falsely) portrays it as a “ringing endorsement.” It is, in fact, far better evidence against the use of cellphone bans than for it.<\/p>\n","protected":false},"excerpt":{"rendered":"

    This article in The Economist entitled \u201cRemoving Phones from Classrooms Improves Academic Performance\u201d (https:\/\/papers.ssrn.com\/sol3\/papers.cfm?abstract_id=5370727) has been cited by Chris Ferguson as an example of how ignoring effect sizes can lead to false conclusions. Here is the abstract of the paper, … Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[3],"tags":[],"class_list":["post-387","post","type-post","status-publish","format-standard","hentry","category-rmstats"],"jetpack_publicize_connections":[],"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/pcjttF-6f","jetpack-related-posts":[{"id":412,"url":"https:\/\/psychexamples.com\/wordpress\/2026\/04\/26\/the-partial-reinforcement-extinction-effect-in-intimate-relationships\/","url_meta":{"origin":387,"position":0},"title":"The Partial Reinforcement Extinction Effect in Intimate Relationships","author":"dbernstein","date":"April 26, 2026","format":false,"excerpt":"A common example of the partial reinforcement extinction effect appears in dysfunctional romantic relationships when one partner behaves badly (perhaps even abusively) toward the other, but not consistently enough or predictably enough to lead the other partner to leave the relationship. Indeed, because the first partner sometimes--but unpredictably--behaves in a\u2026","rel":"","context":"In "Learning"","block_context":{"text":"Learning","link":"https:\/\/psychexamples.com\/wordpress\/category\/learning\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":400,"url":"https:\/\/psychexamples.com\/wordpress\/2026\/04\/25\/the-effect-of-expectation-on-taste-perception\/","url_meta":{"origin":387,"position":1},"title":"The Effect of Expectation on Taste Perception","author":"dbernstein","date":"April 25, 2026","format":false,"excerpt":"In a video found here, the famous magic team of Penn and Teller illustrate how people\u2019s expectations can change their perceptions. Customers at a fancy Los Angeles restaurant were presented with a water menu purporting to list waters from various parts of the world and sold from bottles with elegant\u2026","rel":"","context":"In "Sensation and Perception"","block_context":{"text":"Sensation and Perception","link":"https:\/\/psychexamples.com\/wordpress\/category\/s-and-p\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":150,"url":"https:\/\/psychexamples.com\/wordpress\/2020\/09\/13\/the-effect-of-predictable-vs-unpredictable-stressors-in-fish\/","url_meta":{"origin":387,"position":2},"title":"The Effect of Predictable vs. Unpredictable Stressors\u2014in Fish","author":"dbernstein","date":"September 13, 2020","format":false,"excerpt":"Marine biologists in Portugal observed the physiological and physical responses of European sea bass to predictable and unpredictable stressors. They first taught the fish to associate the sight of a black-and-yellow-striped card with a stressor, namely a mesh net like the ones used to catch fish in aquaculture operations. The\u2026","rel":"","context":"In "Health, Stress, and Coping"","block_context":{"text":"Health, Stress, and Coping","link":"https:\/\/psychexamples.com\/wordpress\/category\/hsc\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":358,"url":"https:\/\/psychexamples.com\/wordpress\/2024\/08\/21\/a-video-game-for-adhd\/","url_meta":{"origin":387,"position":3},"title":"A Video Game for ADHD?","author":"dbernstein","date":"August 21, 2024","format":false,"excerpt":"As described in this article, a video game called EdeavorOTC received FDA clearance as an ADHD treatment in August of 2024. You can find the game here. The assertions made about the game's effects on ADHD should provide your students with a good opportunity for honing their critical thinking skills.","rel":"","context":"In "Research Methods & Statistics"","block_context":{"text":"Research Methods & Statistics","link":"https:\/\/psychexamples.com\/wordpress\/category\/rmstats\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":193,"url":"https:\/\/psychexamples.com\/wordpress\/2021\/07\/20\/classical-conditioning-in-the-classroom\/","url_meta":{"origin":387,"position":4},"title":"Classical Conditioning in the Classroom","author":"dbernstein","date":"July 20, 2021","format":false,"excerpt":"A professor who used a microphone while teaching a large class had the unfortunate habit of occasionally walking in front of the amplifier's speaker, which resulted was a loud screeching sound (US). The first time this happened, the students were startled (UR), and over time, whenever the professor even approached\u2026","rel":"","context":"In "Learning"","block_context":{"text":"Learning","link":"https:\/\/psychexamples.com\/wordpress\/category\/learning\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":334,"url":"https:\/\/psychexamples.com\/wordpress\/2023\/09\/18\/thinking-critically-about-meghan-markles-advice\/","url_meta":{"origin":387,"position":5},"title":"Thinking Critically about Meghan Markle’s advice","author":"dbernstein","date":"September 18, 2023","format":false,"excerpt":"This article from the UK's Daily Mail provides an example of a rather questionable stress reduction device as well as an excellent opportunity for your students to think critically about assertions that are likely to carry extra weight because of their association with an international celebrity, Meghan Markle.","rel":"","context":"In "Research Methods & Statistics"","block_context":{"text":"Research Methods & Statistics","link":"https:\/\/psychexamples.com\/wordpress\/category\/rmstats\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/psychexamples.com\/wordpress\/wp-json\/wp\/v2\/posts\/387","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/psychexamples.com\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/psychexamples.com\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/psychexamples.com\/wordpress\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/psychexamples.com\/wordpress\/wp-json\/wp\/v2\/comments?post=387"}],"version-history":[{"count":1,"href":"https:\/\/psychexamples.com\/wordpress\/wp-json\/wp\/v2\/posts\/387\/revisions"}],"predecessor-version":[{"id":388,"href":"https:\/\/psychexamples.com\/wordpress\/wp-json\/wp\/v2\/posts\/387\/revisions\/388"}],"wp:attachment":[{"href":"https:\/\/psychexamples.com\/wordpress\/wp-json\/wp\/v2\/media?parent=387"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/psychexamples.com\/wordpress\/wp-json\/wp\/v2\/categories?post=387"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/psychexamples.com\/wordpress\/wp-json\/wp\/v2\/tags?post=387"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}