Deep Learning With The Elephants : Planet Money Elephants are in danger. Counting them is crucial to saving them. But they're hard to see in the rainforest. So scientists are enlisting the help of AI technology.
NPR logo

Deep Learning With The Elephants

  • Download
  • <iframe src="" width="100%" height="290" frameborder="0" scrolling="no" title="NPR embedded audio player">
  • Transcript
Deep Learning With The Elephants

Deep Learning With The Elephants

  • Download
  • <iframe src="" width="100%" height="290" frameborder="0" scrolling="no" title="NPR embedded audio player">
  • Transcript


OK, Dina Temple-Raston...



MALONE: ...I think we're going to just start with a single-minded mission that you had in East Africa in Malawi...

TEMPLE-RASTON: To find an elephant.

MALONE: ...To find an elephant and record it in person.



TEMPLE-RASTON: Be up close and personal with an elephant.



TEMPLE-RASTON: So this is our first night. And what they do is they say, look; we're going to go out on the river.

UNIDENTIFIED PERSON: All right, just a quick safety thing for everybody. Please, there are lots of crocodiles in this river, and people are very much on the menu.

TEMPLE-RASTON: And we go on this river at sunset, and the river is just glassy. And everywhere you look, there are animals popping up.

Oh, look; there are hippos there, dropping into the water and coming up and snorting all the air out.

We're completely focused on these hippos.


TEMPLE-RASTON: All of a sudden, these elephants come off the hill and start walking towards the water. And, of course, we weren't rolling.

MALONE: No tape.



TEMPLE-RASTON: Idiot. But the truth is that if you want to be really close to an elephant, you have to go on land.

We saw the fresh dung piles along the road and followed those and found him in this clearing.

Sure enough, right in front of us is this enormous elephant. And you'd think an elephant is so huge that you'd hear him walking all the time. But what they say is the only time you hear an elephant is when he's coming and he breaks a branch, or when he's eating. And what we got was an elephant eating.


TEMPLE-RASTON: It was magical. I was so embarrassed that I found it so magical. But to see them in the wild is magical.

MALONE: Hello, and welcome to PLANET MONEY. I'm Kenny Malone.

TEMPLE-RASTON: And I'm Dina Temple-Raston. Today on the show, a quest to record the sounds of an elephant.

MALONE: Because elephants are being killed at such an alarming rate, experts say they may be extinct in some countries in as few as 10 years. And recording elephant audio is a critical first step in saving the elephants.

TEMPLE-RASTON: We have a story about how researchers tried to pull off a kind of old-school-style heist scheme. And to do that, they had to steal a page from Silicon Valley and Wall Street.

MALONE: There will be fancy gadgets, hiding in trees, secret recordings, but also artificial intelligence. And finally, we will have to understand what a neural network actually does.

TEMPLE-RASTON: Plus, some of the most incredible audio recordings we've ever heard.


MALONE: All right, so I just want to try this thing.


MALONE: Turn your headphones down just a little bit.

TEMPLE-RASTON: Oh, I see (laughter).

MALONE: All right, you ready?


MALONE: I've been practicing this a long time. (Imitating elephant trumpet).

TEMPLE-RASTON: Is that you?

MALONE: That's me.


MALONE: It's me, and it's the best animal impression I can do.

TEMPLE-RASTON: Not bad. And when we think of elephants, that's the sound we think of...


TEMPLE-RASTON: ...That trumpeting sound. But it's basically an elephant shouting.

PETER WREGE: That's right. By far the most frequent kind of vocalization we call a rumble, which is very low-frequency (imitating elephant rumble).

TEMPLE-RASTON: Peter Wrege is an elephant scientist at Cornell University, and he's the director of something called the Elephant Listening Project. It's a team of researchers that works in the rainforests of central Africa.

MALONE: We called Peter, I mean, honestly, a little bit because we wanted to ask him every elephant question we ever had.

TEMPLE-RASTON: Are elephants really scared of mice?

WREGE: Not at all.

TEMPLE-RASTON: That's a Disney thing?

WREGE: (Laughter) That's all from the cartoons.

MALONE: But, also, we called Peter because just a few years ago, there was huge news if you were an elephant researcher.


UNIDENTIFIED REPORTER: A new study, the Great Elephant Census, suggests a failure to protect the world's largest land mammals.

TEMPLE-RASTON: The Great Elephant Census was basically this effort to count every elephant all over Africa because that's the first step in knowing how fast they're being killed off and where specifically they're dying.

MALONE: There was a big caveat to this study, though.

WREGE: The Great Elephant Census depended very heavily on small aircraft. In the savanna, you can fly over it and count herds of elephants. That is not possible to do in the rainforest.

TEMPLE-RASTON: The Great Elephant Census had to leave out the forest elephant, the exact species of elephant that Peter specializes in.

WREGE: You know, the forest elephant is the smallest of the three species of elephant, and their ivory is the most prized of any ivory because it's denser than savanna elephant ivory, and it has a pinkish tinge to it.

MALONE: So Peter and his team set out to see if there is a way to count this elusive forest elephant. And problem No. 1 is, yeah, it's hard to see them from the air. It is also hard to see forest elephants from the ground.

WREGE: Sometimes you see them, let's say, 15 meters away from you on the trail you're walking on. And they move five meters into the forest, and you can't see them, even though they're a 3-ton animal.

MALONE: It's that dense.

WREGE: It's dense, and they - somehow they just disappear (laughter).

MALONE: Masters of camouflage, the elephant.

WREGE: That's right (laughter).

TEMPLE-RASTON: Forest elephants are so hard to count. The best method they came up with was just walking around and counting forest elephant dung piles.

WREGE: There's an estimate of how often does an elephant poop, and how long does a pile of dung last in the rainy season or in the dry season?

TEMPLE-RASTON: This is super time-consuming and pretty limited.

MALONE: So Peter wanted a new way to count these elephants. And he thought, look; if we can't see them easily, and we can't track them easily, what if we tried to just listen for them?

WREGE: If we know how often an elephant calls...

MALONE: Right.

WREGE: ...Then we can spread recorders over a big area...


WREGE: ...And record their vocalizations and use those numbers to count them.

TEMPLE-RASTON: This was an idea that was floated a long time ago by a scientist named Katy Payne. And the technology was reaching a point where maybe it was possible to conduct a kind of acoustic census.

MALONE: So Peter's team decided they were going to do this. First step was to have 50 custom audio recorders made, and they headed to the rainforest in central Africa. Every 5 square kilometers, they placed an audio recorder.

And where would the recorders go? Like, if you just put them on the ground, they'd get stomped.

WREGE: Right. So we put recorders 7 to 10 meters up in a tree, hanging from a tree limb.

MALONE: So seven - I'm going to have to do the math here.

TEMPLE-RASTON: That's 21 to 30 feet up in a tree.

WREGE: We wanted it to be out of the reach of an elephant up on its hind feet because they actually eat that way. They...

MALONE: Were you concerned the elephant might accidentally eat your audio recorder?

WREGE: They have done that (laughter).

MALONE: They've eaten your audio recorder?

WREGE: Well, they've...

MALONE: They've tasted them.

WREGE: They've stuck a tusk through them, yeah.

TEMPLE-RASTON: So Peter and the team climbed 30 feet into the trees, and they strapped these audio recorders in place and then hit record.


TEMPLE-RASTON: Peter had studied these elephants long enough to know that it could be days, weeks even, before an elephant would happen to wander by any given recorder and make that elephant rumble that he was looking for.

MALONE: So the team just walked away from these recorders, and they waited. And after three months, they went back into the forest, back up the trees, and they grabbed the memory cards and brought them home.

WREGE: The most important thing is you get back the sound files and listen to know that, actually, you have a recording.

MALONE: So step one - did it actually record?

WREGE: Exactly. Did it record?


WREGE: When the first data came back from this forest, there is a lot of excitement because every rainforest sounds different - insects calling, frogs calling, the pounding on a tree buttress by chimpanzees.

MALONE: You have that on tape by accident?

WREGE: Oh, we have lots of that.


MALONE: But remember, Peter was listening for elephants, and specifically for that different kind of elephant sound.

WREGE: (Imitating elephant rumble).

MALONE: So they're going through hours and hours and hours of audio.


WREGE: Just like, whoa, we've got it.

MALONE: It sounds like a "Jurassic Park" sound effect. Like, you can't believe that it's a real sound that something makes.

WREGE: Yeah, no, that's very true. And if you have a speaker that can generate power, it shakes your whole body.



WREGE: As soon as I hear it, I'm visualizing this 3-ton beautiful animal, wondering what is it doing when it's doing that? But I think the next thing was actually kind of apprehension because 50 recorders recording 24 hours a day for three months is a lot of stuff to get through.

TEMPLE-RASTON: It's 100,000 hours of audio to get through.


MALONE: Oh, but, Peter, just have a computer do it, I can hear you thinking at home. Well, listening to elephants is exactly the kind of work that has been hard to automate. That's right, it's a human labor story. And after the break, can we teach the robots to listen for elephants?

All right, before we go on, we wanted to just take a quick detour to acknowledge that a lot of elephant stories are a little oversimplified.

TEMPLE-RASTON: And I got a real sense of this when I went to Malawi to try to see an elephant.

Can we ask you, sir? Can we talk?

I was getting a tour of this relatively new fence on the edge of Liwonde National Park when we ran into this local guy.

VERIUS DONZANI: That's - my name is Verius Donzani (ph).

TEMPLE-RASTON: And you live close by?

DONZANI: Yes. My house is 50 meters from the fence.

TEMPLE-RASTON: Before the fence, were there elephants that came around your house?

DONZANI: Oh, very much, especially when they come to a garden where there is food. We had problems.

TEMPLE-RASTON: Problems because elephants are attracted to people's gardens. Verius is a schoolteacher, but like a lot of people on the edge of this park, he relies on his own garden and some nearby fields for his food. And if an elephant came to your garden, it could eat an entire year's worth of crops in just a couple of hours.

Do you have an experience with an elephant...

DONZANI: Yes, I have.

TEMPLE-RASTON: ...That you can tell us about?

DONZANI: It's fortunate that I'm not killed.

TEMPLE-RASTON: Human-elephant conflicts - that's what they're called. And they're really dangerous. Verius remembers this one horrible day before there was a fence here when an entire herd of elephants kind of stampeded through his village.

DONZANI: I tell you it was a disaster. Seven people were killed that day - seven people, one day.

TEMPLE-RASTON: Yes, elephants are magical and incredible when you're a tourist. But if you live next to a bunch of elephants, it's not really so incredible.

MALONE: And this dynamic contributes to the biggest threat facing elephants, illegally killing elephants - poaching. Our elephant researcher, Peter Wrege, says that the smugglers and the international cartels running the ivory trade - what they'll do is that they'll often find a local African farmer - not Verius, but somebody who just had their crops destroyed by an elephant. They'll approach that person to do the actual elephant killing.

WREGE: If, you know, someone comes in and says, hey, I heard you lost everything; go kill that elephant, and I'll give you enough money to buy the food for your family, I don't really blame that guy who went out and did it.

MALONE: Still, Peter is doing everything he can to stop poaching. That's what this acoustic census was all about in the first place.

TEMPLE-RASTON: He and his team had recorded months and months of rainforest audio. And, yes, there were chimps and elephants and rainforest sounds. But as they went through the recordings, they also had gunshots.


WREGE: I don't think we have any where you hear any elephant call or screaming after the shots, possibly because the elephants are actually kind of too far away for the rumble to be heard. Or maybe they were instantly killed, which I doubt. But, you know, just bursts from AK-47s.

MALONE: If Peter's team was going to count elephant sounds for this acoustic forest elephant census, they were also going to count gunshots. This is the best proxy they had for poaching attempts.

TEMPLE-RASTON: Again, the problem was he had too much tape - 50 recorders running for three straight months.

MALONE: It would take a grad student listening to this 24 hours a day six years to go through all of that audio at double speed. Peter needed a better method.

WREGE: You have - I mean, you can look at it with a spectrogram.

MALONE: Spectrogram - this is where you turn sound into a picture. And if you've seen this, it's like a ghostly picture of sound waves. And if you looked at enough elephant sound pictures, you could learn to recognize that.

TEMPLE-RASTON: So a first way to try to count elephant calls is to have a human being sit in front of a computer. And then, like a set of really weird flashcards, up pops a picture of, like, a minute of rainforest sound, and then another, and then another.

WREGE: And that just flashes, flashes, flashes, flashes. And you're looking for that elephant call in there.

MALONE: So that's faster. That's good.

WREGE: It's faster, but doing that with one day of sound takes us about 20 or 25 minutes.

TEMPLE-RASTON: Not super efficient. Ideally, you'd train a computer to do this for you. And that's what Peter's team tried to do.

MALONE: But this is exactly the kind of human labor that's been really hard to replace. Peter's team had to describe to a computer what a picture of an elephant call looks like. But each one is, like, a little different. And sometimes there's a weird-looking one that, you know, a human would be able to point out. But then how do you tell a computer to find those ones? And so Peter's team ended up with this, like, detector program that had really broad parameters of what it was looking for.

WREGE: So it finds signals that aren't actually elephants, and we have to sort through that.

MALONE: Time-consuming.

WREGE: Very time-consuming.

MALONE: Not an ideal solution. Luckily for Peter, another animal researcher had run into this exact same problem.

TEMPLE-RASTON: Let's get this party started, as they say.

MATT MCKOWN: Let's do it.

MALONE: Matt McKown started as a bird researcher.

MCKOWN: As part of my Ph.D. work, I was collecting a ton of recordings - acoustic recordings, you know, sitting on mountaintops with tape recorders. And then I started to explore how I could collect more data.

TEMPLE-RASTON: Just like Peter Wrege, Matt was pretty quickly swamped by the volume of recordings he'd created.

MALONE: And to some degree, this is, like, the modern problem. Our ability to record data of all kinds has outpaced our ability to make sense of that data.

TEMPLE-RASTON: This is what people mean when they talk about the era of big data. A huge part of the tech economy now are companies that are just trying to come up with ways to get through all the stuff that we can now collect digitally.

MALONE: So Matt started looking around for solutions, and he found the buzziest of all Silicon Valley buzz phrases, the neural network.

TEMPLE-RASTON: Talk about neural networks. Explain what those are. Pretend I'm a third-grader.

MCKOWN: Well, you know, basically, what it is - it's a system that's kind of based loosely on the connectivity patterns of neurons in the human brain, right?

MALONE: Right. So, look; there is a lot of really complicated math going on inside of a neural network - so complicated, in fact, that computers couldn't handle it until relatively recently. But we're going to do our imperfect best right now to just give you a sense of what is actually going on inside of a neural network.

TEMPLE-RASTON: Let's say you want to show a computer a picture and have it tell you, dog, not dog.

MALONE: That is really easy for a human being to do, but it's really hard to write a set of rules for a computer to identify a dog. You have to be like, you know, it's like a dog. I don't know. It's, like, this tall usually. And it's fluffy, but, I mean, not always. I don't know. It is just too much to have to explain.

TEMPLE-RASTON: So the idea behind a neural network is it's a computer program where you don't have to explain everything about a dog. But in order to make that program work, you have to give your neural network a bunch of pictures and say, this dog, this not dog.

MALONE: Yeah, you're showing it pictures with a dog and without a dog and telling it which is which. And what is happening each time this program looks at a picture is that you've got this first layer of what people call neurons. These are metaphorical neurons. But each of these neurons is going to focus on some general facet of the dog image. So imagine there's one neuron that's looking at these and thinking, like, huh, you know, when there is a dog, I'm seeing, like, a lot of brown a lot of the time.

TEMPLE-RASTON: Well, I'm noticing a lot of curves in these dog pictures.

MALONE: And these neurons - they start to learn, you know, like, if I see brown...

TEMPLE-RASTON: If I see curves...

MALONE: ...Then they are going to light up.

TEMPLE-RASTON: And then the later layers of metaphorical neurons refine the analysis. Oh, hey, look; I see spots.

MALONE: I don't know. I got, like, a tail - like, a tail-looking thing over here.

TEMPLE-RASTON: Next time these neurons see that stuff...

MALONE: They're also going to light up as well. And this whole process is cumulative. The idea is that dogness becomes so decentralized in this network that no matter what picture you show it, if it's a dog, you're going to get some combination of neurons lighting up and being like, yeah, yeah, yeah, yeah, this is a dog.

TEMPLE-RASTON: Again, to train the network, you have to give it tons and tons of pictures and say, this dog, this not dog.

MALONE: And with each new picture of dog, the program teaches itself a little more about dogness, and it starts to hone in on the teeny-tiny traits of dogness that are most important to look for. And the network is almost certainly going to find ways to identify a dog that we humans wouldn't have even thought to tell the computer to look for in the first place.

TEMPLE-RASTON: This is what people mean when they say machine learning.

MCKOWN: And this whole field is called deep learning. Because of the advent of sort of modern computer technology, you can make these neural networks that have many, many layers of neurons, and so you can start to recognize real fine-scale patterns.

MALONE: Matt McKown ended up starting a company to help researchers like Peter Wrege set up their own neural network to dig through their data, and his company is now working with Peter and the Elephant Listening Project to deal with all that rainforest sound.

Now, the way this works is that Peter Wrege's rainforest audio gets turned into those spectrograms, pictures of sound. And then Matt McKown is training a neural network to go through every single rainforest sound and say, elephant sound, not elephant sound.

TEMPLE-RASTON: So 50 recorders, three months each, over 100,000 hours of audio - Matt says his neural network will be able to get through all of that in just four hours.

MALONE: Just four hours to count up all of the elephant sounds so Peter can finally know, like, how many forest elephants are there, and where are they? It may be hard to tell a computer exactly what to look for, but a neural network can learn to look for this.


WREGE: That, actually, is a fantastic example of two females who are performing what we call a greeting ceremony.

MALONE: Again, Peter Wrege. And he says these rumbles - these are a perfect example of elephant language. They've heard this pattern before when elephants have been separated for some time and then they come together to, like, greet each other again.


WREGE: I think it's very much like if you run into a friend on the street, you know, that you haven't seen for a while. It's a back-and-forth, whoa, how are you? And, oh, I'm OK. What about you? Oh, it's not so good. I've lost my job. Oh, my God. You know, who knows what they're really saying? But it's that - perhaps that kind of thing.


TEMPLE-RASTON: When Peter set out to do this audio census, that neural network was still being trained. And today, it's still being trained. They're still showing it tons of elephant call images.

MALONE: Yeah, it takes a long time. So for the first round of data, Peter went ahead and used the old-school method of telling a computer what an elephant call looks like. And, yes, it's less precise. It takes more labor. But it worked. And the idea as a whole, recording forest elephants to track forest elephants, worked.

TEMPLE-RASTON: Peter's team was able to find all those elephant calls, place them on a map, figure out what time they'd happened. And if you imagine, like, a TV weather map, instead of rolling clouds, it's elephants. And it's the kind of thing you need to make anti-poaching teams as effective as possible.

WREGE: You can say, OK, we know that elephants are not using this huge part of this park for these seven months. We don't need to send any anti-poaching teams there because no poachers are going to find an elephant anyway. And you can see where the population of elephants is moving across the landscape at different times of year.

MALONE: Oh, that's cool, yeah.

WREGE: And no one has ever known that for any elephant of any species anywhere.

TEMPLE-RASTON: This has Peter excited for when his neural network really trains up. That's when the possibilities really get going.

MALONE: A neural network would move way faster and be way more accurate. And you could imagine this network learning to hear different kinds of elephant calls, like when it's a distress call or maybe some other sign of danger.

WREGE: And we have two gunshots here, or we have AK-47 shots here. If that can be fed out of the rainforest in real time, then the anti-poaching people know where to go to intercept that poacher that just killed an elephant.

TEMPLE-RASTON: Do you think that AI is going to save the elephant?


WREGE: I actually do, yes.


MALONE: And, Dina, I think we should play one last elephant call from Peter Wrege's recordings.

TEMPLE-RASTON: OK, let's do it.


MALONE: That is a real elephant trumpet. And I just - I want to say, like, I feel like my impression is very good. All right, here's mine (imitating elephant trumpet). And here's the real one.


TEMPLE-RASTON: (Laughter).

MALONE: It's good. It's good.

TEMPLE-RASTON: OK. I got to tell you I was a little dubious, but when I hear them side by side, it's a lot better than I thought it was.


MALONE: If you liked this episode of PLANET MONEY, it helps us if you just send it to somebody. Or if you'd leave us a rating in your podcasting app, those are some of the best ways you can support our show.

TEMPLE-RASTON: Special thanks this week to the Elephant Listening Project and to Conservation Metrics and John Keefe.

MALONE: John is part of the Quartz AI Studio, and he is incredibly helpful. We always love to get your feedback. We are on Facebook, Twitter and Instagram, @planetmoney. Or you can always send us an email,

And this episode that you just heard is part of a larger project that Dina Temple-Raston is working on. It is about technologies that are watching us - things like artificial intelligence, data surveillance - and how those things are changing the way we live. Dina, you want to explain a little bit more?

TEMPLE-RASTON: The project is called I'll Be Seeing You, and it's coming to NPR in September, so I hope you'll keep an ear out for it.

MALONE: Today's episode was produced by Nick Fountain, Darian Woods and Michael May. Bryant Urstadt edits our show. Alex Goldmark is our supervising producer. This is PLANET MONEY from NPR. I'm Kenny Malone.

TEMPLE-RASTON: And I'm Dina Temple-Raston.

MALONE: Thanks for listening.


Copyright © 2019 NPR. All rights reserved. Visit our website terms of use and permissions pages at for further information.

NPR transcripts are created on a rush deadline by Verb8tm, Inc., an NPR contractor, and produced using a proprietary transcription process developed with NPR. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.