How to properly interpret statistics. : The Indicator from Planet Money Statistics and the information we get from them have a massive influence on our worldviews and the decisions we make, but how can we ensure we're interpreting them properly? Today, we find out.

Making Sense Of Pandemic Stats

  • Download
  • <iframe src="https://www.npr.org/player/embed/956471751/956836360" width="100%" height="290" frameborder="0" scrolling="no" title="NPR embedded audio player">
  • Transcript

SYLVIE DOUGLIS, BYLINE: NPR.

(SOUNDBITE OF DROP ELECTRIC'S "WAKING UP TO THE FIRE")

CARDIFF GARCIA, HOST:

Hey, everyone. It's Cardiff. This is THE INDICATOR FROM PLANET MONEY. Think back for a second over all the big questions about the coronavirus pandemic that have applied to you and to your loved ones. How infectious is the virus? How fatal might it be if you catch it? What's the best way to protect yourself? How effective are the vaccines likely to be? Scientists obviously need to know how to use statistics to run the experiments to answer these questions. But it's also important for the public to understand how to interpret the answers, how to see the data clearly, to know what to believe and not to believe. A lot of times people just don't have the confidence to know. They're not sure.

Tim Harford is an economist and the author of a new book that's coming out soon called "The Data Detective: Ten Easy Rules To Make Sense Of Statistics."

TIM HARFORD: The idea of the book is to give people that confidence to help them distinguish between what's true and what's not true, because there's basically no way to figure out certain truths about the world without statistics.

GARCIA: Today on the show, a conversation with Tim about how these rules have applied to the pandemic and how they can help guide us through the rest of it.

(SOUNDBITE OF MUSIC)

GARCIA: Jim Harford, welcome back to THE INDICATOR. How are you?

HARFORD: I'm well, Cardiff. It's great to chat to you again.

GARCIA: Tim, one of the earliest lessons in your book is that when we're looking at statistics, we should first examine our own feelings, because if we really desperately want to believe something, it's easier for us to end up being misled by bad statistics or we might end up ignoring good statistics. And either way, we'll end up just being wrong. So how do you think that lesson has applied in the COVID pandemic?

HARFORD: Yeah. It's so interesting. I felt it was really important to emphasize the significance of our emotional reaction, that we believe things that we want to believe. We believe things that our friends tell us, that people on our side of the political debate believe. And so that's why I began "The Data Detective" with this slightly surprising piece of advice, which is before you get into any spreadsheets or calculators or anything, just think about how you're feeling. What's your emotional reaction? Are you feeling defensive? Are you feeling vindicated? Are you feeling this proves you right? What was interesting about the coronavirus pandemic is, at first, there wasn't any of that emotional reaction. I mean, people were scared, but people were just - they weren't interested in winning an argument. They wanted to understand what was going on. So I'd come out of the Brexit debates in the U.K., the polarization in the U.S., where nobody wanted to give the other side of the argument a second's worth of serious consideration. And suddenly along comes this pandemic and all of these scary numbers. And people are like, oh, hang on. I don't want to prove anybody wrong. I don't want to win an argument. I just want to understand, what is this thing? How dangerous is this thing? It didn't last long, of course. People pretty soon polarized even on the subject of COVID. But there was this moment and I remember it.

GARCIA: Yeah. And, Tim, another point you emphasize is that when it comes to the use of statistics, trust with the public is hard to build, but easy to lose if there's a mistake. And there were mistakes early in this pandemic when we were all just kind of operating on really limited information. It was a kind of fog in which we were trying to figure out what exactly was happening. And I remember that at first, the Centers for Disease Control here in the U.S. said that masks were not helpful. And then later on it changed its mind when it looked again at the data more closely. But it was a mistake.

HARFORD: Yeah. It's a tough one because we need an answer now, right? And the answer might be wrong. So there is a cost to putting out a public statement that says, don't worry about masks, probably not a big deal, and then having to go back on that statement and say, we were wrong. There's a cost to that. There's also a cost to not telling people anything or not being able to provide any information. So there have been some very impressive data gathering efforts. There's a huge trial called the RECOVERY Trial, which I think is organized here in the U.K., where they just said, look, there are all these different possible drugs - there's monoclonal antibodies, there's dexamethasone, there's hydroxychloroquine, there's all this stuff - and we don't know what works. So we're going to do proper clinical trials really fast. People are dying in hospital. We're going to figure out what can save their lives. And within weeks, just weeks, we had really good data. And we were able to say, for example, that the steroid dexamethasone is highly effective, and we saved quite a lot of lives. And hydroxychloroquine might have worked, but it turns out it doesn't. That sort of thing matters. And now, of course, we're desperate for the vaccine. And the same questions arise. Is the vaccine safe? Is the vaccine effective? And we've spent quite a lot of time just waiting for the data to come in. So you can move too quickly on this, but you can move too slowly as well.

GARCIA: Yeah. On the subject of vaccines, recently, there has been a push in the U.K. and now the U.S. to start giving as many people as possible their first doses of a vaccine instead of holding back some of the vaccine, so that people who get their first dose can then get their second dose a few weeks later. And the idea here is that giving more first doses also offers more people at least some immunity. But the results of the clinical trials that have been done so far were based on two doses, so the evidence is still sort of limited. And so this is a decision that also has to be made with imperfect data. And it's become a really, like, heated thing for some people.

HARFORD: Yeah. I mean, you can see it very clearly on an individual basis. I get emails. I present a program for the BBC about the vaccination rates now called "How To Vaccinate The World." And people email me and they say, oh, it's outrageous. The British government has switched to this kind of first dose first policy. And my mother was going to get her second dose, and now she has to wait for her second dose. And I'm thinking my dad's 77, he hasn't had his first dose yet. I don't write that in response because I totally understand where they're coming from. But for everyone who's complaining they haven't got their second dose, there's someone else who's waiting for their first dose. These are not straightforward problems. What I think we should do, by the way, is we don't have the evidence base, but we could get it really quickly. If we ran some some quick, rigorous, randomized trials, we could find out whether delaying the second dose for, say, three months causes a problem or doesn't - my guess is not. That's what the - my reading of the evidence. But so far, the only way to find out is to run a randomized trial. And we could do that ethically if we wanted to, if we could be bothered to get our act together.

GARCIA: Yeah. Tim, this is also just a really good example that shows why it's so important to have good data, but also that there are real people behind the data, behind the numbers.

HARFORD: Yeah. One of the early chapters in the book emphasizes the challenge of trying to combine your personal experience with the data and not privilege one over the other. Most people would be like, oh, I believe what I see with my own eyes and you know, but - that's evidence to me. And they're not interested in what's in the spreadsheets. But I think that geeks like you and me - and I mean that in the most sincerely admiring way...

GARCIA: Yes, thank you (laughter).

HARFORD: ...We maybe tend to privilege the spreadsheet a bit too much and to value the data a bit too much. And data and statistics properly gathered can tell us a lot, but they can also be rather thin and miss a lot of detail. And the best view, the wisest view comes when you're able to synthesize your own experience, your personal experience, the human stories behind the data, but also what's in the spreadsheet. You need both to really understand the world.

GARCIA: All right. Tim Harford, thanks for being on THE INDICATOR.

HARFORD: Thank you.

GARCIA: Tim's book, "The Data Detective: Ten Easy Rules To Make Sense Of Statistics," is out in the U.S. in early February, but it is already available to preorder. This episode of THE INDICATOR was produced by Jamila Huxtable and fact-checked by Sean Saldana. THE INDICATOR is edited by Paddy Hirsch, and it is a production of NPR.

Copyright © 2021 NPR. All rights reserved. Visit our website terms of use and permissions pages at www.npr.org for further information.

NPR transcripts are created on a rush deadline by an NPR contractor. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.