Predicting The Future: Fantasy Or A Good Algorithm? The intelligence community has always been in the business of forecasting the future. The question is whether tapping into publicly available data — Twitter and news feeds and blogs — can help them do that faster and more precisely. Now, a cutting-edge tech company is trying to use data to predict seminal events before they happen.

Predicting The Future: Fantasy Or A Good Algorithm?

  • Download
  • <iframe src="" width="100%" height="290" frameborder="0" scrolling="no" title="NPR embedded audio player">
  • Transcript


We move now from a story where we know the results to one about predicting them. Intelligence analysts are always tried to anticipate events - figure out the future by gathering secret information using wiretaps or satellite images, but it's sometimes just as valuable to gather and think carefully about publicly available information - the word on the street. So maybe it's not surprising that a company would seek insight and future predictions by mining Twitter feeds, news reports and blogs.

NPR's Dina Temple-Raston explains.

DINA TEMPLE-RASTON, BYLINE: You may have seen this old Tom Cruise movie.


TOM CRUISE: (as Anderton) There hasn't been a murder in six years. The system, it is perfect.

UNIDENTIFIED MAN: Chief Detective John Anderton leads the nation's most advanced crime force.

CRUISE: (as John Anderton) I'm placing you under arrest for the future murder of Sara Marks.

TEMPLE-RASTON: The future murder of Sara Marks, he says. The movie "Minority Report" is all about police getting information and stopping crimes before they actually take place. That movie came out 10 years ago. Now the U.S. intelligence community is trying to anticipate events like protests or uprisings or oil price hikes before they happen.

A Swedish-American start-up called Recorded Future is trying to help.

CHRISTOPHER AHLBERG: What we're trying to do here at Recorded Future is figuring out a cool way that we can observe the world. Trying to find new ways of actually generating data, what's going on in the world - what did happen, what will happen.

TEMPLE-RASTON: That's Christopher Ahlberg, the company's CEO. He says there are hints about the future everywhere. Governments release economic projections. Newspapers report on upcoming events. Twitter is a great source too. In Egypt last year, organizers used Twitter or social media to rally protestors. So that's part of what makes Recorded Future different. It's developed an algorithm that allows it go through billions of pieces of information and sort it by time.

AHLBERG: An event might be a person traveling from A to B to people talking to each other, a government guy making a statement, a country doing a military maneuver - all of those sort of activities that can be pretty, you know, small scale or very large scale, and then time associated with that.

STEVE SKIENA: Predicting the future is a hard game. I guess it was Yogi Berra who said that predicting is hard, especially about the future.

TEMPLE-RASTON: That's Steve Skiena, a computer scientist at Stony Brook University in New York. He developed similar technology for another start-up.

SKIENA: The Recorded Future idea seems to be that if you can get ideas about what other people are saying is going to be happening in the future for certain kinds of events, it's probably very good information.

TEMPLE-RASTON: Events like political unrest, for example. In January 2010, Recorded Future predicted that Yemen was headed for disaster. Why? A combination of floods, famine and Islamic terrorists. The company used Twitter feeds, blogs, U.N. food program data, and news sources to come up with their forecast. A year later, thousands were in the streets of Yemen protesting food prices. A short time after that, Yemen's president fell from power. Of course what Recorded Future didn't predict was the Arab Spring. Still, predicting the future by trolling the Web may be possible. But there are skeptics.

GARY KING: We need to know whether you are getting the right answer randomly or you're getting the right answer because of something you're doing.

TEMPLE-RASTON: Gary King is a professor from Harvard and the director of the Institute for Quantitative Social Science. King is considered a guru in this field.

KING: It would be a miracle if you got it right all the time. Picking one prediction out of a hundred and showing that it was consistent with the future is essentially irrelevant.

TEMPLE-RASTON: And if you're right 50 percent of the time, is that good or is that luck?

KING: That sounds like luck.

TEMPLE-RASTON: King says there are one billion new social media posts every two and a half days. But it's easy to draw the wrong lessons from all that data. For example: after the attack on the U.S. consulate in Libya, Ahlberg, the CEO of Recorded Future, sent me a prediction that the attack would eventually be traced to al-Qaida's arm in Yemen, the same al-Qaida group that had tried to blow up U.S. airliners. Why did Recorded Future think this? Because its algorithm linked a militant group in Libya, Ansar al-Sharia, with an al-Qaida group in Yemen with the same name. The problem: They share the name, but no other connection. Ahlberg said it happens, but he wouldn't put a number on the company's prediction track record.

AHLBERG: When we do our blogging that we do, it's not meant to be NPR and it's not meant to be sort of the academic sort of thing. We use that to show highlights of things.

TEMPLE-RASTON: The people who use Recorded Future, he said, are experts. They would catch that kind of mistake.

AHLBERG: The unfortunate aspect of being called Recorded Future is that people expect to push a button and find, you know, the future of everything. And I'm like, look, we're trying to provide data, demonstrate data, show data, visualize data in a way that makes smart people smarter.

TEMPLE-RASTON: Ahlberg has about 100 subscribers and at least two very important financial backers: The CIA's investment arm, In-Q-Tel, and Google Ventures. They have reportedly poured millions into the company. Maybe they see something about the future that the rest of us can't. Dina Temple-Raston, NPR News.

Copyright © 2012 NPR. All rights reserved. Visit our website terms of use and permissions pages at for further information.

NPR transcripts are created on a rush deadline by an NPR contractor. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.