School’s out

Across much of the world, one of the initial responses to the covid-19 pandemic was to close schools. WIth the virus still circulating, many jurisdictions continue to grapple with the question of when and how to resume physical classes. Much of the political debate is focused on the health risks. Many are understandably concerned about how safe it is to send students and teachers back into classrooms. But history also points to the risks associated with maintaining shutdowns. Specifically, disrupting children’s schooling can have persistent effects on their overall level of schooling.

Can we evaluate the consequences of mandated school closures? Plainly, one can only guess at the long-term effects of a coronavirus-induced shutdown: in the absence of time travel, the lifetime outcomes of today’s kids is unknowable. But maybe past examples from other health crises can help to gauge the direction and magnitude of effects?

In new research, Meyers and Thomasson (2020) analyse one historical case involving widespread school closures: the 1916 polio epidemic in the US. Thanks to comprehensive immunisation, polio is today eradicated across the developed world. But no vaccine (or effective treatment) was available in the early 20th century. The 1916 epidemic was significant, with more than 23,000 reported cases distributed across just about every state in the US.

There are two plausible channels by which the polio outbreak might have affected student outcomes. The direct channel is through infection: the health complications arising from polio might have impaired the learning potential of those struck by the disease. The indirect channel is through public health measures intended to contain the virus’ spread: for example, school closures.

Meyers and Thomasson hypothesise that the effects of each of these channels might vary with age. Polio disproportionately affected children up to 10 years of age: thus, these children were relatively more susceptible to the direct effect of contracting the disease. By contrast, the effect of the epidemic on children aged 14 years and above was more likely to be observed in higher drop-out rates: the absence from school prompting teenagers to start work rather than continue with education.

A good thing we eliminated polio

The headline finding is that the polio epidemic resulted in reduced educational attainment for individuals that in 1916 were either up to 10 years of age or between 14 and 18 years of age. (There is no statistically significant effect for those who were aged between 10 and 14 during the epidemic — consistent with the story that this cohort was less susceptible to the virus, and were too young to drop school entirely.)

As the table below shows, the authors test three specifications of the model with different configurations of fixed effects and controls (summarised in the author’s table notes, and discussed in greater detail in the paper). The outcome being tested is years of education, as measured in 1940 (that is, looking at adults in 1940, some of whose schooling might have occured during/around the epidemic). The age cohorts (based on age in 1916) are, as described, 0-10; 11-13, 14-17 and 18-21. The latter group is the benchmark: they were too old to have been in school, but are otherwise sufficiently similar to their younger peers, that their results should be comparable. That is, comparable apart from the 1916 polio outbreak. Specification 3 is the preferred model.

The key results are found in the first three lines of the table: the interaction terms for polio and the affected age cohorts. Interpreting this is a little complicated. “Polio” here refers to the number of diagnosed polio cases per 1000 residents (“morbidity”); the age cohorts are binary dummy variables (1 for an individual’s applicable age group, otherwise 0). The interaction term means that, for every unit increase in the number of polio cases, an individual of age 15 (captured in the first age group listed in the table) in 1916 received 0.16 fewer years of education relative to the benchmark reference group (specification 3).

To take an example: the lowest recorded state-level polio morbidity rate per 1000 residents in 1916 was 0.003 (Montana), the highest was 1.376 (New Jersey). Thus, white males who were 15-year old students in Montana in 1916 received, on average, 0.00048 (= 0.003 × 0.160) fewer years of schooling than their 20-year old peers. By contrast, the equivalent cohort of 15-year old boys in New Jersey in 1916 received, on average, 0.22016 (= 1.376 × 0.160) fewer years of schooling than their 20-year old peers. (The paper separately explores the effect on white female students, while data on non-white students are considered unreliable.)

To the layperson, the effect sizes here might not seem like much. But recall that these are results averaged across a cohort of students. The likely story here is that a higher proportion of students (relative to those aged 18-21 in 1916) dropped out of school early rather than that the cohort as a whole received less education. For example, in New Jersey, one could surmise that slightly more than one in five students in the 14-17 age group in 1916 dropped out of school a year earlier. (Or alternatively, that around one in ten students dropped out two years earlier. We can’t say with certainty based on the results provided. This is merely about possible interpretations of the average effects.)

So, what about school closures?

Does this mean that shutting down schools during a public health crisis is a “bad” thing for students’ education? Maybe. But that’s not a conclusion one can draw from these results. (Nor is it, to be a clear, a conclusion that Meyers and Thomasson attempt to draw!)

First, the results here bundle together the direct and indirect channels: that is, we can say that a polio epidemic affects education outcomes. But this is both due to the direct effect on students who fall ill, and the broader context that might result in some students choosing to leave school early.

Second, the indirect channel in this case is not solely mandated school closures. The results pick up the effect of the collective package of public health measures instituted during the epidemic, rather than school closures specifically. Moreover, it is probable that public fear of the epidemic also contributed to changes in behaviour and attitudes, which would also be captured in these results. The authors explicitly acknowledge this. The point is that many students did not attend school; whether that was due to closures or alarmed parents keeping their kids at home is not a focus of this study.

This highlights a key conceptual challenge in measuring causal effects of specific measures. If parents — in the absence of a policy decision to close schools — would have refused to send their kids to school, the causal effect on education outcomes of school closures would be considerably more limited.

Even setting this distinction aside, understanding the effect of disrupted schooling (whether due to government policy or parental decisions) on student performance would require partialling out the direct effects of the polio virus. Most directly, you might seek to disaggregate student results at the individual level — that is, identify which students contracted polio, and track education choices and student performance with respect to individual polio exposure. But such micro-level data in this historical case are not available.

Lessons for today?

Covid-19 is not polio. Though knowledge of this new virus is still relatively limited, health experts suggest that children are not especially vulnerable to covid-19; the current pandemic poses the greatest risk to the elderly and those vulnerable due to other health conditions. Thus, relative to polio, it seems less likely that children who fall ill with covid-19 will experience significant direct-channel effects on their education performance. (Though as other research on influenza shows, another relevant question here might be in-utero effects: to what extent children born to women who contract covid-19 during pregnancy might experience life-long effects.)

It is also harder to draw conclusions about the possible indirect-channel effects. A pertinent difference between 1916 and 2020 is the potential for remote learning. Though schools have been physically closed, teaching has continued over the internet. The extent to which e-learning is an effective substitute for traditional classroom-based teaching is a separate question. (Anecdotally, there seems to be little argument from teachers, students and parents alike that the virtual classroom is inferior to the physical one.)

Given the observed persistent effects from the 1916 polio epidemic, one might expect that any difference in education quality associated with the rapid switch to remote learning also has the potential to manifest itself in lifetime effects for the current corona cohort. Equally though, awareness of the problem might also trigger targeted interventions that support today’s students.

In short, one can safely conclude that major epidemics can have adverse, long-term consequences on education. Venturing into less certain territory, it might be the case that school closures have negative effects on student performance. But these effects in a modern context may well be alleviated by remote learning. In any event, rushing to reopen schools might not achieve much if parents choose to keep their kids at home.