Higher Expectations (Haidt's After Babel)

Haidt set high expectations and we should demand high standards right from the start: the formulation of hypotheses and methodology.

​

The introductory essay Welcome to the After Babel Substack is mostly the beloved writing by Haidt that so many of his readers — including those of us who fundamentally disagree with him on some issues — have come to appreciate: clear and insightful yet engaging commentary on social issues that truly matter.

Good writing, however, is no substitute for science — otherwise Just So Stories would be textbooks in high school biology classes.

Scientific Methodology

Haidt’s series of articles on Substack is in preparation for his planned book with the tentative title Kids In Space: Why Teen Mental Health is Collapsing.

Haidt sets high expectations in The Teen Mental Illness Epidemic Began Around 2012 when he writes:

In sum, it’s reasonable to start with skepticism of my claim (with Jean Twenge) that there is an epidemic of mental illness that began around 2012, and that is related in large part to the transition to phone-based childhoods, with a special emphasis on social media. It makes sense to embrace as a null hypothesis the skeptics’ view that there is nothing to see here, just another moral panic, and the kids are fine. I am in full agreement that the burden of proof falls on me. 

Unfortunately, it is here that Haidt’s writing loses clarity and that his allusion to scientific methodology gets muddled.

Haidt conflates two hypotheses: that there is a crisis and that he knows the causes of this crisis. This conflation is reinforced in the next paragraph:

But if you take that as your null hypothesis, then you should be open to evidence that the null hypothesis is false and this time is different. Anecdotes about kids who began cutting themselves the week after going on Instagram won’t do. You’ll want to see peer-reviewed studies and high-quality surveys showing 1) that there is in fact an epidemic of mental illness and 2) that phones and social media are substantial contributing causes.

One may of course agree that there is a crisis (this time is different) and yet completely reject the blame placed by Haidt on smartphones and social media.

The mention of a null hypothesis — a technical term related to statistics testing — is also unfortunate. At best, such use adds nothing but a false sense of scientific rigor.

In this case, however, it is outright misleading. This is because Haidt, in a subsequent article responding to his critics (Why Some Researchers Think I’m Wrong About Social Media and Mental Illness), defends himself as follows:

The skeptics are demanding a standard of proof that is appropriate for a criminal trial but that is inappropriately high for a civil trial or a product safety decision.

I’m actually fine with a lower standard of proof in safety decisions involving kids, but then the rejection of a null hypothesis is not an apt analogy, as it typically requires at least 10 to 1 odds against the hypothesis being wrong.

In general, it is best to refrain from the use of technical terms in analogies.

What is The Scope?

Lack of clarity is present already in the precis of the planned book, given that its subtitle is Why Teen Mental Health is Collapsing and yet its declared ambition is more limited:

My goal is to answer the question: What happened to Gen Z? Why did their levels of anxiety and depression skyrocket around 2012?

Does this mean that Haidt has no plans to address the doubling of adolescent suicide?

And does it mean that Haidt will not be explaining subsequent developments, such as the even stronger increases in depression and depressive symptoms after 2017?

This is all the more puzzling given that Haidt declares much stronger ambitions only a few paragraphs later:

It’s about what we’ve done to Gen Z, and how to stop doing it and restore healthy childhood. The primary prescription: 1) more unsupervised time to play and explore (before puberty), in order to learn to become self-governing; and 2) less and later access (after puberty) to social media and the screen-based life. [Kids In Space Overview]

Will Haidt consider healthy childhood to be restored if depression and anxiety rates decline and yet adolescent suicide rates of girls and boys remain at double the level they were in 2007?

And what about other mental disorders that are shown on the graphs that Haidt includes in his book Overview:

Is Haidt’s prescription for a healthy childhood intended to redress the stark increases in ADHD, anorexia or schizophrenia?

Perhaps Haidt is concerned only with widespread disorders: depression and anxiety.

That is fine, but if so then Haidt needs to say so.

Haidt also needs to clarify if victim indoctrination — the belief that hurt feelings are intrinsically harmful — needs to be countered if we are to restore healthy childhoods.

Furthermore, Haidt needs to reconsider the role of screen time. If the problem is only social media and not screen time, as most of Haidt’s recent writing seems to suggest, then it is unclear how his prescription for a healthy childhood will work, especially for boys. If parents limit social media but not screen time, how will unsupervised time help if boys and girls remain home watching Netflix and TikTok or Youtube videos?

Finally, Haidt needs to acknowledge that proving a cause of harm does not suffice to justify prescribed treatment. This is because the treatment could be ineffective or even harmful. Haidt, therefore, needs to separate his ‘causes of harm’ hypotheses from his prescriptions hypotheses and weight the evidence for each separately.

​

Conclusion:

Haidt needs to clarify the scope of his concerns and enumerate all the primary causes of harm as well as all the primary treatments of harm. Haidt also needs to clarify the expected methodology for proving each of the hypotheses involved (including the efficacy of the proposed treatments).

​

Notes:

Flexibility: it is of course perfectly fine for Haidt to modify his theory as required in the course of gathering more evidence gaining a better understanding of the issues. At this point, however, he needs to be at a stage where the main contours of his theory are already drawn.

Null hypothesis: the rejection of the null hypothesis is, strictly speaking, limited to the role of chance in sampling or another, specified, source of uncertainty or noise. The issue of what sources of uncertainty and noise need to be considered, and how they should be evaluated, can be quite complex (e.g. consider the role of the placebo effect when formulating the null hypothesis for a treatment).

TikTok and YouTube: are TikTok and YouTube social media? I do not think so, given that their primary use is the passive consumption of videos. This is fundamentally different from Facebook or Twitter, where users are forced to take at least a few initial steps toward their social presentation and active participation is expected. In any case I see no justification for classifying TikTok as social media unless we do the same with YouTube.