GUY RAZ, HOST:
Today on the show, Ideas About Big Data - how to make sense of it and how it's transforming the way we live, our understanding of the world and beyond. And when it comes to the study of our universe, the data hasn't always been that big. So in 1929, Edwin Hubble proved the universe was expanding. And he proved this using measurements from just 24 different galaxies.
ANDREW CONNOLLY: That's right. He showed that the further the galaxies were away, the faster that they were expanding. From this came the idea that the universe itself was expanding.
RAZ: This is astronomer Andrew Connolly.
CONNOLLY: I work on the interaction between telescopes that survey the sky and big data.
RAZ: Andrew says the idea that everything in the universe was expanding gave rise to another idea, which was the Big Bang, a big idea from that small data set of 24 galaxies. And then 70 years after Hubble, something similar happened...
CONNOLLY: That's right.
RAZ: ...When two teams of scientists were studying supernovae.
CONNOLLY: Supernovae are exploding stars.
CONNOLLY: The final throes of a star explodes and brightens, so you can see them to great distances. And so we could measure how far away those supernovae were. And so what they found - these supernovae were slightly further than they expected and that meant the universe had to be slightly larger than expected which meant that the universe had to be accelerating.
RAZ: The expanding universe was accelerating. Now, this idea was crazy but true. And it came from studying just 42 exploding stars.
CONNOLLY: But I think it took them about three years over time to collect those 42 supernovae.
RAZ: That sounds like a really long time for such a small dataset.
CONNOLLY: Well, they had to look through thousands of galaxies to find those 42 supernovae, but it's not a lot of data.
RAZ: OK, so what if there were a way to collect more data? I mean, even our best telescopes today are really good at looking at specific parts of the sky but not that good at taking it in all at once. They sort of treat the universe more like a static image than a moving picture. And as Andrew Connolly explained in his TED Talk, that's a problem.
CONNOLLY: The universe is anything but static, constantly changes on time scales of seconds to billions of years. Galaxies merge. They collide hundreds of thousands of miles per hour. Stars are born. They die. They explode in these extravagant displays. In fact, 10 supernova per second explode somewhere in our universe. If we could hear it, it would be popping like a bag of popcorn. But the telescopes we've used over the last decade are not designed to capture the data at their scale. So this is driving us to new technologies and new telescopes, telescopes that can go faint to look at the distant universe, but also telescopes that can go wide to capture the sky as rapidly as possible.
RAZ: What Andrew Connolly is describing is a brand new kind of telescope, one that he and a team of scientists are working on right now on a remote mountaintop in Chile, and it's a telescope that uses big data, unlike any telescope before. So instead of measuring 42 supernovae over three years...
CONNOLLY: We should be finding about 500 to 1,000 supernovae every night.
CONNOLLY: So millions within 10 years.
RAZ: I mean, it means that the rate of discovery is about to explode.
CONNOLLY: The rate of discovery will explode and scales with the amount of data that we're collecting.
RAZ: And the telescope making this all possible - it's called the large synoptic survey telescope.
CONNOLLY: Possibly, the most boring name ever for one of the most fascinating experiments in the history of astronomy.
RAZ: Or LSST for short.
CONNOLLY: We're building the LSST. We expect it to start taking data by the end of this decade. I'm going to show you how we think it's going to transform our views of the universe because one image from the LSST is equivalent to 3,000 images from the Hubble Space Telescope, each image three and a half degrees on the sky, seven times the width of the full moon. Well, how do you capture an image at this scale? Well, you build the largest digital camera in history. Using the same technology you find in the cameras in your cell phone or in the digital cameras you can buy in the high street, but now at a scale that's five and a half feet across, about the size of a Volkswagen Beetle. So if you wanted to look at an image in its full resolution, just a single LSST image, it would take about 1,500 high-definition TV screens.
On this camera, we'll image the sky taking a new picture every 20 seconds, so every three nights we'll get a completely new view of the skies above Chile. Over the mission lifetime of this telescope, it will detect 40 billion stars and galaxies, and that would be for the first time we'll have detected more objects in our universe than people on the Earth.
Now, we can talk about this in terms of terabytes and petabytes and billions of objects, but a way to get a sense of the amount of data that will come off this camera - is that it's like playing every TED Talk ever recorded simultaneously 24 hours a day, seven days a week for 10 years. And to process this data means searching through all of those talks for every new idea and every new concept looking at each part of the video to see how one frame may have changed from the next. And this is changing the way that we do science. But as we enter this era of big data, what we're beginning to find is there's a difference between more data being just better or more data being different, capable of changing the questions we want to ask.
RAZ: Here's why the LSST and all of the data it will collect can help scientists ask bigger questions about our universe. When you or I look up at the sky at night. We're not just looking into space, we're actually looking back in time.
CONNOLLY: Because the nearest star is - what? - four - about just a little bit more than four light years away. So the light you see from that is actually what that star was producing four years ago. If we're looking out at near - nearby galaxies like Andromeda, we're looking back a few million years, to the most distant galaxies where we're looking back billions of years. Astronomers are almost like the ultimate historians.
RAZ: It's so crazy to think that this enormous telescope is going to be capturing all of this stuff that's already happened.
CONNOLLY: Yeah. Millions of years ago, billions of years ago.
GUY RAZ, HOST:
So what kind of mysteries could we solve with all the data that we're going to get that we couldn't get before?
CONNOLLY: So the aspect about this telescope is that it's not designed just to answer one question. It's designed to provide a census of our universe. And so that means that we can understand the expansion of our universe, how rapidly it's expanding, what is it that's driving that expansion, what is dark energy, dark matter, the distribution of stars within our own galaxy, how our galaxy, the Milky Way, grew. We'd begin to understand the solar system, right?
CONNOLLY: The properties of objects within the solar system, you know, how our solar system formed, how water came onto the planet Earth. No matter how much data we get, there's always new questions to ask.
RAZ: But how do you - how do you make sense of all that data? I mean, you said looking at one photo from the LSST would take, like, 1,500 HDTVs. I mean, how do you even begin to look at a photo like that?
CONNOLLY: Well, we can't. I mean, when we actually process the data, when - so we have thousands of computers. So as the data comes off that camera, it gets swept into thousands of computers that are processing and calibrating and are detecting objects. It's like if you took an image of a crowd and you wanted to pick out all the faces in the crowd...
CONNOLLY: ...All the people in the crowd or to find the pictures where two people are closest together. That's how we process the data. And we have to be able to look at the images not with our eyes but with algorithms and find anything that's moved or change and characterize it. And these images are public, and that means not just to astronomers. That means that if you looked at one of these images, you may be the first person to see that part of the sky.
CONNOLLY: It took three years to find just 42 supernovae because the telescopes that we build could only survey a small part of the sky. With the LSST we get a completely new view of the skies above Chile every three nights. In its first night of operation, it will find 10 times the number of supernovae used in the discovery of dark energy. This will increase by a thousand by within the first four months, 1.5 million supernovae by the end of its survey, each supernova testing which theories of dark energy are consistent and which ones are not until hopefully at the end of this survey, around 2030, we expect to hopefully see a theory for our universe, a fundamental theory for the physics of our universe, to gradually emerge.
But if looking through tens of thousands of galaxies revealed 42 supernova that turned our understanding of the universe on its head, when we're working with billions of galaxies, how many more times are we going to find 42 points that dont quite match what we expect? What's so exciting about the next decade of data in astronomy is we don't even know how many answers are out there, answers about our origins and our evolution. How many answers are out there to questions that we don't even know that we want to ask? Thank you.
RAZ: Does this mean, Andrew, that we don't have a whole lot left to learn from, you know, like, the old version of the telescope, like the kind I look through with my son in my backyard? Or is the universe so complicated now that big data is the only way to understand this?
CONNOLLY: No, I think big data adds into that. So big data allows us to identify interesting objects that then you - we may take individual telescopes to target to look at them in much more detail. In fact, the fact that these data are public means that the LSST could announce the discovery of an object and you or your son could go and look at that object with your backyard telescope, make a measurement and tell us what it is. And so I don't think big data is the only way. I think big data enables much more science. And these data will let us understand how we formed, how our solar system formed, how, you know, we came into being - to understand our place in the universe.
CONNOLLY: Andrew Connolly is an astronomer at the University of Washington. His TED talk is at ted dot com. And the LSST - the Large Synoptic Survey Telescope - is expected to be completed by 2019.
COLORAMA: (Singing) There's too much data, too much data. You're filling up my head with too much data.
RAZ: Hey, thanks for listening to our show on big data this week. If you want to find out more about who was on it, go to ted dot npr dot org. If you want to see hundreds more TED Talks, check out ted dot com or the TED app. Hey, thanks for listening to our show about failure this week.
Our production staff here at NPR includes Jeff Rogers, Brent Baughman, Meghan Keane, Neva Grant and Sanaz Meshkinpour, with from Rachel Faulkner and Daniel Shukin. Our partners at TED are Chris Anderson, Kelly Stoetzel and Janet Lee. If you want to let us know what you think about the show, you can write us at tedradiohour at npr dot O-R-G. And you can follow us on Twitter. It's at TEDRadioHour. I'm Guy Raz, and you've been listening to ideas worth spreading right here on the TED Radio Hour from NPR.
(SOUNDBITE OF TED TALK)
(SOUNDBITE OF TED TALK)
(SOUNDBITE OF TED TALK)
(SOUNDBITE OF TED TALK)
(SOUNDBITE OF SONG, "TOO MUCH DATA")
NPR transcripts are created on a rush deadline by Verb8tm, Inc., an NPR contractor, and produced using a proprietary transcription process developed with NPR. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.