Haidt: Social Media Associations with Harm
Haidt offers only one study showing elevated risks for girls; furthermore, his argumentation is problematic and contains serious errors.
Haidt devotes section 3 of his article Social Media is a Major Cause of the Mental Illness Epidemic in Teen Girls to associations.
Incredibly, in the entire section Haidt offers only one — one! — data set that reveals elevated risks of mental disorders for heavy users of social media among girls.
Fuzzy Counts
Instead of citing specific papers, Haidt starts with counts based on his list of associative studies in Social Media and Mental Health: A Collaborative Review:
The great majority of studies find a positive correlation between time on social media and mental health problems, especially mood disorders (depression and anxiety). At present, there are 55 studies listed in our review that found a significant correlation, and 11 that found no relationship, or nearly no relationship.
No!
Many, perhaps most of the studies on the list do not examine associations between time on social media and mental health problems.
By my count the majority of the studies listed are about about screens or digital devices or the Internet, not specifically about social media.
Also, many of the social media studies do not measure time or frequency (e.g. they measure problematic behaviors), and many are not about associations with mental health disorders (as promised by the article title) — e.g. one measures social isolation, another social skills, and so on.
Finally, many, perhaps most of the studies, seem to be about adults, which is problematic given that the article needs to provide evidence about adolescent girls.
Fuzzy Patterns
Haidt continues:
In fact, there is a revealing pattern found across many studies and literature reviews: Those that look at all screen-based activities (including television) for all kids (including boys) generally find only small correlations (usually less than r = .10), but as you zoom in on social media for girls the correlations rise, sometimes to r = .20, which is quite substantial, as I’ll show in a moment.
Haidt provides no evidence of this supposed pattern. Furthermore, the mentions of r — which Haidt leaves undefined — make no sense here, since the letter r is typically reserved for measures of linear predictability, not for measures of risk.
Haidt continues:
The general finding in these correlational studies is a dose-response relationship such as the one in Figure 2, from Kelly, Zilanawala, Booker, & Sacker (2019)
Once again Haidt provides zero evidence for his assertion.
At least he finally cites a paper — Social Media Use and Adolescent Mental Health: Findings From the UK Millennium Cohort Study by Y. Kelly et alia — and it is an important study, although its cope is limited: it concerns only 14-year-old UK girls circa 2015.
So far we have nothing else from Haidt that truly matters.
Boys and Girls
Haidt presents the following graph based on data from the Kelly study:
The graph appears to be accurate, but under it Haidt states:
The dose-response effect is larger for girls. For boys, moving from 2 to 5 hours of daily use is associated with a doubling of depression rates. For girls, it’s associated with a tripling.
No!
Moving from 2 to 5+ hours is associated with doubling for both boys (14.5/6.8 = 2.1) and girls (38.1/18.1 = 2.1) — see Table 2 of Kelly.
Later in section 3.3 Haidt writes:
there is a consensus that the relationships are tighter for girls; see Kelly, Zilanawala, Booker, & Sacker (2019), Nesi & Prinstein (2015), and Twenge (2020).
Well, the ‘Kelly’ here is the same as the ‘Kelly’ above.
N&P found the correlation between frequency of use and depressive symptoms to be 0.09 for boys and -.002 for girls — the opposite of ‘tighter for girls’ (see Table 1).
And Twenge? That paper discusses the same data as does Kelly.
So what consensus?
The Size of the Association?
In section 3.4 Haidt says there is a consensus that “the size” of the association between social media use and mood disorders is at least r = .15 — and that it is therefore a large association.
This is double nonsense. First, there is no single “correct” effect size for the complex relationship between social media use and mood disorders — not even approximately. Second, standardized predictability (r) is not a measure of risk; without additional information it is in general useless for evaluations of potential harms.
Haidt then cites r = .20 for girls in a paper he co-wrote. What data this paper examined? Oh, the same UK Millennium Cohort Study data examined by Kelly.
So once again we are back to the same survey data.
Haidt continues:
It is much larger than the correlation between childhood exposure to lead and adult IQ (which was found to be around r = .11).
This comparison is utterly meaningless. Neither values of r can be interpreted without additional information. And of course there is no single value that alone characterizes the association between childhood exposure to lead and adult IQ.
BTW, r = .11 played no role in evaluation of lead effects in the paper Haidt cites — it’s never mentioned in the Abstract and in fact it is never directly mentioned anywhere within the entire paper (it’s just one of the correlations in Table 2 that provides auxiliary information).
Social Media Definition
Haidt never defines what he means by social media, but he practically equates it with Instagram and Facebook as those are the only ones he mentions repeatedly. In that sense even the Kelly paper is problematic as it relies on time spent on social networking where the survey question explicitly mentions WhatsApp, a messaging tool.
Discussion
Haidt needs to define what he means by social media, cite and discuss 3 or 4 best studies showing elevated risks for adolescent girls (like the paper by Kelly), and list a dozen or so additional studies that found elevated risks. Haidt should then discuss if there are any important studies that contradict his hypothesis.
Conclusion
Haidt cites one good study where 5+ hours of social networking by 14-year-old UK girls circa 2015 is associated with more than doubling the risk of clinical depression.
That’s all — nearly everything else in section 3 (Associations) is either irrelevant, unsubstantiated, vague, or wrong.
Notes:
Mismeasure of Risk
Throughout the article, Haidt repeatedly mention various values of ‘r’ without specifying what correlation the r measures and why it should matter. It seems Haidt thinks there is a single correct method to measure “the strength” of the association between social media and mood disorders and that this “strength” tells us how dangerous it is. Both notions are misconceptions.
Orben & Przybylski
Much of the time in section 3 is spent by Haidt dealing with a 2019 paper by Orben & Przybylski. Nearly everything Haidt states about this paper is irrelevant to the task of providing evidence that social media harm girls; the few remarks of Haidt that are relevant happen to be egregiously wrong. I will deal with this matter in a separate article.
I do think that depression is associated with heavy Instagram use (though I suspect this relationship is primarily due to troubled girls gravitating towards the platform).
One recent evidence, albeit not peer-reviewed, comes from a 2022 Common Sense survey of 1,300 U.S. adolescent girls (How Girls Really Feel About Social Media).
In the survey, 21% of girls were found to have mild symptoms of depression and another 17% of girls was found to have substantial symptoms of depression.
Among Instagram users, girls with substantial symptoms were much more likely (24% vs 9%) to use Instagram “almost constantly” (see Table 9). This 24% rate, however, is similar to that of YouTube (22%) and Snapchat (24%) and below that of TikTok (34%) — so Instagram does not appear particularly more addictive for depressed girls than other major platforms routinely referred to as social media.
Mild symptoms were not, curiously, associated with constant use:
What I notice in all these studies is that there seems to be almost no attempt to describe the process by which screen users, or social media users, become depressed. I mean, I watch NBC Nightly News on Youtube, and I suppose I could choose to experience that as depressing; I choose instead to find it disconcerting. Similarly, I could find the challenge of sorting out and interpreting the data in Haidt's arguments to be so frustratingly difficult as to be depressing; instead I just find myself losing interest.
In other words, events don't cause emotions; people choose them, either consciously or unconsciously. Maybe it makes more sense to help them to make better choices, than to solve the problem by removing their access to the world.