The State of California and its 58 counties have issued more than 1,500 public health orders since the beginning of the COVID-19 pandemic — including San Francisco and LA County’s recent orders to once again shut down restaurant dining, indoors and out, as the number of cases spike again throughout the state.
The orders have chiefly called for moratoriums on big gatherings, the closures of in-person schooling and non-essential businesses, the use of face masks, and contact-tracing efforts.
So how effective have those public health orders been? Did shutting down restaurants and schools curb the spread of the epidemic in the Golden State? Have strict mask mandates in some counties been more effective than closures in others?
Many policymakers and public health officials want answers. The stakes are high, given the need to protect the public while addressing faltering economic activity in the midst of closures.
Academic researchers are attempting to address these questions, but to do so, they need fine-grained information on the types and strictness of policies put into effect, along with their timing.
Members of the Stanford-CIDE Coronavirus Simulation Modeling Consortium — organized and led by Stanford Health Policy’s Jeremy Goldhaber-Fiebert and other Stanford faculty and researchers along with their colleagues at CIDE in Mexico — knew that for their own work on modeling COVID-19 epidemics for all counties in California, they also needed this data.
So the SC-COSMO team undertook an open-source effort to address this data gap. The team has just released a huge dataset consisting of over 1,500 public health orders from January through November. They have made the data publicly available on their website, as well as MedRxiv, an open source medical research website for pre-peer reviewed studies and public comment. And they developed a data visualization tool that allows users to easily visualize and compare information within and across counties.
Stanford’s SC-COSMO team has for months been partnered with the state to provide data for the California COVID Assessment Tool, or CalCat, which has helped hospitals and public health departments determine their next steps as they monitor the current spread, what’s expected in the two-to-four weeks ahead of them — and the long-term impacts under different scenarios.
The data have become particularly important to CalCat, which has shown a steady increase in the spread of the coronavirus since mid-October. Gov. Gavin Newsom said Monday the state had recorded over 100,000 new cases in just the last week and that if the trend continued, "California will need to take drastic action."
“Stanford’s new health order dataset helps the State of California understand the course of COVID and plan the ongoing response,” said Ryan McCorvie, a statistician working for the California Department of Public Health’s COVID-19 modeling group. “Analysis of the detailed local response in each county can help policymakers across the state judge outcomes effectively.”
The two-dozen member Stanford team — from SHP faculty to medical and graduate students and contributors in Mexico — has listed all the public health orders and identified hundreds of website URLs for each county on which public health orders were released.
“The dataset allows researchers to ask questions about how effective public health orders are at reducing epidemic transmission or how much economic impact various orders may be having,” said Goldhaber-Fiebert, an associate professor of medicine.
But it’s hard to answer those questions without county-level data to determine how useful and strict — or not — the orders have been.
“People are also using our data to do more precise forecasting on what is likely to happen in the coming weeks,” he said.
One of those is Yaser S. Abu-Mostafa, a professor of electrical engineering and computer science at the California Institute of Technology. He directs Caltech’s national COVID-19 trajectory model, which is using the Stanford dataset.
“As we discussed this model with city and state health officials in California and New York, one of their most frequent requests has been reliable modeling of the effect of public health policies on the spread of COVID,” said Abu-Mostafa. “For months, our efforts in this regard have been stymied by a lack of comprehensive data on what policies had been enacted, and Dr. Goldhaber-Fiebert’s work has given us access to this unique and crucial dataset — and we hope to use it to fuel data-driven policy analysis to help public officials make crucial decisions.”
Goldhaber-Fiebert noted the team has seen patterns in how counties may be using combinations of public health orders.
“For example, in some counties, strictness of business closure tracks face mask strictness, which may indicate additional efforts to control epidemic transmission,” he said, “whereas in other counties, as business closures become less strict, face mask strictness increases. This may indicate attempts to offset the disruption of such closures with other measures to control transmission perhaps in settings where incidence was somewhat lower.”
Erin Holsinger, a Stanford pediatrician who worked on the project, said the dataset would allow health-care providers to easily determine the coronavirus burden in their patients’ home county, complementing information on prevalence of cases with the public health restrictions that exist in the counties where they’re from.
“Physicians and hospitals are often caring for patients who come from a wide and diverse geographic area,” Holsinger said. “This information will allow physicians to make lifestyle recommendations for their patients that are tailored to their community's risks and restrictions."
SHP Research Data Analyst Elizabeth Long, a member of the modeling team, notes most of the discussion of public health orders has focused on state-level mandates.
“But California is a state with a large population and diverse urban, suburban, and rural areas each with different incidence rates,” she said. “The county-level public health orders data can help improve models and therefore help policymakers create targeted, protective policies to slow the epidemic.”