It looks like the modelers have way overshot the original prediction. Before we get into that, let me explain some terms in the statistical world. First of all, there is the term, “statistics”, which is a very broad term, encompassing just about everything in the, well…… statistical…. world. A subset of statistics is ‘probability’, i.e. “the probability of rain is 40%.” Related to probability is a word I think should get a little more hype is that of inferencing. To make an inference is to reach a conclusion based on the evidence you have (along with some common sense, or intuition).
When a modeler makes a statement, “we believe that 200,000 Americans will die from COVID-19, then they are making an inference. It appears (fortunately) that it will fall well short of the mark. Does it mean the inference is “wrong”? George Box is known for this expression, “All models are wrong, but some are useful.” What exactly did he mean? That, by and large, none are going to be perfect. But some will be more accurate than others.
Getting back to the under prediction of COVID-19, certainly, there have been mitigating factors, such as social distancing. I believe many experts made a prediction of a few hundred thousand with those factors in mind. So, they are still off the mark. They are being maligned by many people for making a poor prediction. Is this fair? Really, it is hard to say. Sometimes things happen that are really hard to envision. Take a football team, say the San Francisco 49ers last year from last year, when they went 13-3, and came within a play of two of winning the Super Bowl. The year before they went 4 and 12. What was the prediction for them going into last year? Well, everybody might have a different model, some may have thought their coach, Kyle Shanahan, was only good as an offensive coordinator, and would fail as a head coach. Some maybe thought he learned from his mistakes, and his players would play hard for him, and he would be a “boy genius”. Many were in the middle of that spectrum. What would Jimmy Garoppolo do for a full year? Was he overrated? How good was their draft? What about injuries? There are a million things that go into a prediction. Many prognosticators figured they would be improved, perhaps to as much as .500, maybe 9 and 7. I do not think the greatest clairvoyant could have seen anything close to what they ultimately did. Does mean everybody who had some kind of a model (a lot of us have very “loose” models) had a poor model? Not really. We can make a bad prediction, but it still is a relatively strong model. Were the models (or modelers) that over predicted what would happen with COVID-19 poor? Unless we know what went into the models (which only the modelers hold that information), we will never know. But I do believe that the general public has been overly tough on these forecasters, not quite understanding that, much like with predicting Frisco to go 8 and 8 with the info they had, 200,000 deaths might have been appropriate given the information.