Haidt: Longitudinal Evidence

Haidt provides barely any longitudinal evidence at all that social media might be harming girls.

Haidt attempts to present longitudinal evidence in section 4 of Social Media is a Major Cause of the Mental Illness Epidemic in Teen Girls.

Unfortunately the very title of section 4 is misleading:

4. Question 2: Does Social Media Use at Time 1 Predict Anything About Mental Health at Time 2?

Such questions are actually answered by correlational studies.

The crucial point of longitudinal evidence is to determine if it is possible to predict changes from time 1 to time 2.

Indeed Haidt himself implies this by later writing that “you can look to see if an increase or decrease in some behavior at one point in time predicts a change in other variables at the next measurement time” — that is what differentiates longitudinal studies from mere correlational studies.

Causation

Unfortunately Haidt precedes this passage with the assertion that “these studies have an interesting property that allows researchers to infer causality“ — which is in general not true!

The best we can do is assert the absence of credible alternatives to one specific cause.

Haidt makes an even stronger assertion about causation shortly afterwards:

For example, let’s say a teen reports that she spends two hours a day on social media, on average, across a 10-week study. If in week 3 she suddenly reduces her time to zero, what do we expect to happen to her mood in week four? Will she be happier or sadder? If there is, on average, a change in happiness the week after people quit or reduce their social media time, then we can infer that the change in mood was caused by the change in behavior the prior week.

This again is not true. It is in general not possible to deduce such causation, because the actual cause might have been a third factor affecting both social media and happiness (assuming it was not a mere coincidence). For example, her social media time might have been reduced because her dad confiscated and sold her smartphone and her happiness continues to plummet a week later because her dad continues to make her life a living hell!

In the very last sentence of section 4, Haidt states:

But 73% of the studies that looked for causal effects a month or more in the future found them.

This is not true — again, it is in general impossible to deduce causation in merely longitudinal (rather than experimental) studies.

A Count and a Pattern

The entire evidence presented by Haidt amounts to one count and one pattern: most studies on his list of longitudinal papers found a link and the ratio increases with the time lag between measures.

Haidt ascribes this pattern to social media being addictive; in reality, there might be a simpler explanation: perhaps the short-term studies have fewer participants and therefore lesser power to detect statistically significant associations.

Haidt never discusses a single study on his list, essentially telling readers it is their responsibility to decide which studies are truly important, relevant, and correct.

The List

As with the list of correlational papers, the longitudinal list includes studies that do not involve adolescents, do not measure specifically social media use, or do not measure mood disorder indicators.

The inclusion of some studies as positive evidence is questionable.

For example, Are Mental Health Effects of Internet Use Attributable to the Web-Based Content or Perceived Consequences of Usage — study 2.1.2 on Haidt’s list, implies in its Longitudinal Regression Analysis section that social media changes (‘Socializing’) did not predict changes in the Depression Anxiety Stress Scale.

Another study, Longitudinal associations between changes in screen-time and mental health outcomes in adolescents (2.1.5 on Haidt’s list), reports:

Associations between changes in screen-time and psychological difficulties were inconclusive and only television/DVD use was found to be significantly associated with this outcome.

It is unclear to what degree can readers trust the classifications of studies on Haidt’s list.

Discussion

Haidt is like a preacher who asks people to believe something and when questioned why, he just directs them to read the Bible — they’ll find the answer somewhere in it!

As with correlation studies, the first step should be for Haidt to cite and explain at least 3 longitudinal papers that provide the best evidence and that actually involve adolescents and indicators of mood disorders.

Conclusion

Haidt fails to present anything close to a convincing argument about longitudinal studies and essentially shifts the burden of finding and examining the relevant evidence to his readers.