Jane M Fraser

What’s new?

My colleague Bill Thomas of EJB Partners called my attention to an article published by McKinsey & Company on June 25 titled “Demystifying modeling: How quantitative models can – and can’t – explain the world.” Highlighting the role of modeling during the COVID-19 crisis, the four authors describe the powers of models and the pitfalls to avoid when using models. The article is really excellent and I recommend you go read it before finishing my piece.

What does it mean?

Models come in many forms: mental models, physical models, mathematical models, simulation models, and more. A model is a representation of reality that can be analyzed to derive conclusions that improve one’s understanding of reality. I hesitate to make this sweeping statement, but the ability to make and use models, as a type of tool, seems to me to be a definition of human thinking abilities. For 40 years, I taught engineering students the various models that engineers find useful in designing objects and systems to make the world better – from the equation F=ma to the M/M/1 queuing model. One of the most important phrases I taught them was: “It’s only a model.”

The McKinsey & Company is short (go back and read it if you haven’t yet). Given the article’s focus on COVID-19 models, you also might want to look at this tool that my state of Colorado has made available for citizens to explore the effect of different behaviors on spread of the disease.

While short, the McKinsey & Company covers all the important points about models. Models are useful in clarifying which drivers matter, determining how much an input can matter, and facilitating discussions about the future. A model can’t fix bad data, assumptions and simplifications must be examined, and users should not expect too much certainty.

“The purpose of modeling is insight, not numbers” is a quote attributed to many people; I first heard it attributed to cybernetics expert Ross Ashby. Experiments can be performed on a model more easily, more cheaply, and more quickly than on the real world, where you may only get one chance to see, for example, how the COVID-19 pandemic evolves. Insight means an understanding of how the various parameters interact to create the outcome, but insight is not a forecast. As baseball expert Yogi Berra said, “It’s tough to make predictions, especially about the future.” The McKinsey & Company authors, in a sidebar, describe how they use scenarios, which are not meant to be forecasts, but are meant to support discussion of the implications.

What does it mean for you?

I would only fault the McKinsey & Company article for omitting the politics and power involved in creating a model, and the fact that people need to trust the modelers, not just the model, if they are not to  reject the model’s findings that don’t agree with their existing beliefs. You need to ask questions about any model: who made it and what assumptions went into it, but also what are the explicit and implicit goals of those who made the model.

The Colorado tool is open source, with the code posted on Github, a repository for software, especially open-source software. The documentation tab provides a link to more information, including the names and qualifications of the people who created the model. Other links point to a more detailed description of the model including assumptions such as: the incubation period is 4.2 days with 1 day before that for presymptomatic infectiousness, 1800 ICU beds are available in Colorado, recovered individuals are assumed to remain immune to infection, and no cases of COVID-19 are imported or migrate from outside Colorado. Changing the parameters of this model quickly convinced me that modest improvements in social distancing (three parameters) and the proportion of population wearing masks (one parameter) could crush the epidemic in Colorado by September. I was very surprised by how small the necessary changes are. “Wear a damn mask,” Governor Polis said recently. The model fits my existing beliefs, but others may not be persuaded. I am not here to debate, only to note the people who make the models are often seeking to make a point, whether in the public arena or in a private company. The creators of this Colorado model certainly want people to practice social distancing and to wear face masks.

One goal in creating a model is always a robust model, that is, a model with conclusions that do not vary much if the key assumptions of the model are tweaked. For example, what happens to the results if the incubation period is 3.2 or 5.2 days instead of the assumed 4.2 days? Also, the Colorado model assumes a single number for that parameter, the same for everyone; more sophisticated models would use probability to model that parameter. A model has to be stressed by using sensitivity analysis: how much in migration of COVID cases would be needed to change the conclusion I reached above that only small changes in social behavior are needed to make us safe?  Unfortunately, I see those cars with Texas license plates, although I also saw them a lot before COVID-19.

The key to the use of a model is to let the modelers argue and make them revise the model again and again. “What if …” should be the starting words in almost every question you ask. This model is overly simple, meant to be used by the public, but more sophisticated models are available.

A model is simply an extended argument, a case made to support conclusions. The argument may be in equations or a computer program, but it is an argument and can be examined and questioned just as one can do with any argument. The strength of a model is that it is explicit, even if sometimes complicated. A model, like Colorado’s COVID-19 model, can also be used to generate scenarios, which in turn can be used for planning. Colorado’s health care system can plan for a worst case scenario.

The problem with any model is that it is only a representation of reality, not reality itself. I taught my students to say “It’s only a model,” said with a shrug. A model is a representation of reality, and thus fails to capture some aspects of reality.

Where can you learn more?

The forecasts of some more sophisticated COVID-19 models are summarized here by the website fivethirtyeight and here by the CDC. The McKinsey & Company article has a sidebar with four suggested articles to learn more about COVID-19 modeling.

An editorial at this link discusses the differences between and potential use of two COVID-19 models. The article makes wise recommendations about the use of models. In particular, the point of analysis is to support decision making about action, and models can recommend different, but reinforcing actions to take.