DAVID GREENE, HOST:
OK, raise your hand if you've been on the Google Flu Trends site today. OK, raise your hand if you even knew the Google Flu Trends site existed. Well, it does. And if you go to the site today, you'll see that flu season seems to be winding down. There's a somewhat less rosy picture from the Centers for Disease Control and Prevention website though. That's where the official flu information is reported, though it's a couple of weeks old.
Google started its flu site in 2008 to provide more up-to-date information. But a report in Science magazine finds that quicker isn't necessarily better.
NPR's Richard Harris reports.
RICHARD HARRIS, BYLINE: A few years ago, scientists at Google realized that they could measure flu activity by tracking when people searched for flu-related terms. They created a handy site called Google Flu Trends. And it works pretty well, some of the time, but don't bank on it.
DAVID LAZER: It missed by a huge amount last year. And actually, as it turns out, it's been missing by a fair amount for several years.
HARRIS: David Lazer is a professor of political science and computer science at Northeastern University. He says Google Flu Trend whiffed during the 2013 flu season.
LAZER: This is like bases loaded and bottom of the 9th and, you know, striking out on three pitches. They predicted twice as many flu cases as the CDC later said there were.
HARRIS: Lazer has written a critique of Google Flu Trends in the latest Science magazine. He finds that the data collected painstakingly from around the country and forwarded along to the CDC, is still much better - even with the time lag.
LAZER: You could just have used old CDC data, let's say two or three weeks old, and have projected forward and done a better job than Google Flu Trends.
HARRIS: Is this a useless exercise then?
LAZER: Not at all. Not at all. I think actually the core idea is a terrific one.
HARRIS: Lazer thinks Google could improve its system with the help of outside scientists, if it were less secretive about what exactly what it's doing to get its results. But he sees some inherent issues, as well. Google is always refining its search methods, which is good for people doing Google searches, but not so good for analyzing that data - consistency is important in science.
Andrea Dugas, who's an emergency room physician and a professor at the Johns Hopkins medical school, says Google Flu Trends is going after an important problem.
ANDREA DUGAS: If we knew ahead of time, OK, flu is going to, you know, really peak in the next few weeks, we need to start getting additional resources to help manage all these patients that are going to be coming in.
HARRIS: During past outbreaks, Hopkins has actually opened up new areas of the hospital to care for flu patients. Dr. Dugas and her colleagues have been working to improve those predictions, looking at various different methods to do that.
DUGAS: The most accurate of those models was the one that looked at the confirmed flu cases. That one was the most accurate in predicting what was going to happen in the following week.
HARRIS: When they added in the Google Flu Trends data, their prediction got slightly better.
DUGAS: Adding that in really kind of helped refine the model and give us a better prediction. But the main driver for that was the number of flu cases.
HARRIS: She and colleagues developed an app for hospitals to use, to predict the ebb and flow of flu. Dr. Dugas says Google's approach is no substitute for lab tests, hospital reports and on-the-ground data. And it's important to note that it's not tracking actual flu, just searches of common symptoms like fever, cough and sore throat.
Google didn't provide a scientist to comment for this story. But a Google report notes that part of the issue was that they sometimes get a flood of searches simply when there's a lot of flu in the news. So they updated the flu tracker system this past fall to help reduce the errors that result from that.
Richard Harris, NPR News.
(SOUNDBITE OF MUSIC)
GREENE: This is NPR News.
(SOUNDBITE OF MUSIC)