Following Digital Breadcrumbs To 'Big Data' Gold In the past couple of years, computing, storage and bandwidth capacity have become so cheap that it's altered the scale of what's possible in terms of collecting and analyzing data at every turn. It's a tectonic shift that will continue to affect many things we do for decades to come, one expert says.
NPR logo

Following Digital Breadcrumbs To 'Big Data' Gold

  • Download
  • <iframe src="" width="100%" height="290" frameborder="0" scrolling="no" title="NPR embedded audio player">
  • Transcript
Following Digital Breadcrumbs To 'Big Data' Gold

Following Digital Breadcrumbs To 'Big Data' Gold

  • Download
  • <iframe src="" width="100%" height="290" frameborder="0" scrolling="no" title="NPR embedded audio player">
  • Transcript


Here is a small sample of companies that rely on massive amounts of data to design their products: Facebook, Groupon, the biotech firm Human Genome Sciences. It's hard to comprehend the amount of information that can be harnessed and crunched, whether it's consumer behavior or genetic sequences. Those enormous data sets are called Big Data. As the term suggests, they're huge in scope and power. In this first of two stories, NPR's Yuki Noguchi reports on big data's impact on many industries.

YUKI NOGUCHI, BYLINE: To understand how Big Data works, think about your daily life. You write an email, call your boss, pass a security camera. Maybe you buy a plane ticket online. Taken alone, this is disjointed, boring information. To Elizabeth Charnock, it makes up your digital character.

ELIZABETH CHARNOCK: Digital character is this idea that almost everybody these days leaves behind a giant, digital breadcrumb trail.

NOGUCHI: Charnock founded Cataphora, a company that can process huge amounts of this sort of data about employees, to determine patterns. She says those patterns can predict everything from a person's mood to their skill as a manager, to a person's inclination to commit fraud. Take rogue trader Jerome Kerviel, who cost his French bank billions of dollars in losses.

CHARNOCK: His cell phone bill was literally an order of magnitude larger than any of his co-workers. Why? Well, because he wanted to put less things in writing. He almost never took vacation, even though French people love to take vacation.

NOGUCHI: Charnock says Kerviel also circumvented usual trading and communication protocols.

CHARNOCK: Any one of those things, you kind of say, so what? But what we look for is a number of them that on the surface, perhaps, don't seem to be related but all seem to be happening at the same time.

NOGUCHI: And Charnock says had the French bank analyzed that data, they might have flagged their rogue trader earlier. But big data is not just about connecting dots for criminal detection. The ability to process so much information, and process it so quickly, makes all kinds of things possible that weren't before.

So LinkedIn finds jobs or people you might like to know about. And biotech companies can analyze gene sequences in billions of combinations, to design drugs. Data analytics itself is not new. Two decades ago, Wall Street hired teams of physicists to analyze investments.

But in the last couple of years, computing, storage and bandwidth capacity have become so cheap that it's altered the scale of what's possible. Now, with very little money, a gifted student or a small start-up can design big data applications.

CHRIS KEMP: Everywhere you look, there's an opportunity to collect more data, and then apply a statistical or mathematical approach to understanding what's happening.

NOGUCHI: Chris Kemp is CEO of Nebula, a firm that provides storage and computing capacity for other companies to be able to process their big data applications. He says ultimately, Big Data will give consumers better tools so they can do a better job of predicting things like prices - whether an airfare is likely to go up or down. Farmers can do a better job of ensuring their crops if they can forecast the weather with greater accuracy.

Oren Etzioni teaches computer science at the University of Washington. He says this trend is fueling intense demand for mathematics and computing talent.

OREN ETZIONI: We have seen the industrial revolution, and we are witnessing a data revolution.

NOGUCHI: Etzioni started three big data companies. One of them,, employs four Ph.Ds to design better programs to forecast prices on consumer electronics. Etzioni says a good data scientist can write algorithms that filter data, understand what it's telling you, then graphically represent it. The end result is like getting a bird's-eye view of a vast territory of information.

Big data can, and occasionally does, go wrong. Comic examples of that include mismatched recommendations, like my TiVo thinks I'm gay. But think about a company divulging your Web surfing history - with your name attached - and you begin to get a sense of how big data opens the door to new possibilities of security or privacy breaches.

James Slavet is a venture capitalist at Greylock Partners. He says his firm invests in companies that use big data creatively and responsibly. He says data does not stand in for human judgment.

JAMES SLAVET: They do use it to make the judgment kind of more sound, more objective and to, hopefully, lead to better decision making.

NOGUCHI: Slavet calls big data a tectonic shift, one that will continue to affect many things we do, for decades to come.

Yuki Noguchi, NPR News, Washington.

MONTAGNE: And tomorrow, we'll hear from Yuki about how the popularity of big data is creating recruiting wars for math talent. For more on the big data series, visit

Copyright © 2011 NPR. All rights reserved. Visit our website terms of use and permissions pages at for further information.

NPR transcripts are created on a rush deadline by Verb8tm, Inc., an NPR contractor, and produced using a proprietary transcription process developed with NPR. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.