SACHA PFEIFFER, HOST:
Scientists say they found a new way to decode the stream of words and sentences in our heads. NPR's Jon Hamilton reports on a study that used MRI scans and artificial intelligence to transform a person's brain activity into phrases.
JON HAMILTON, BYLINE: Researchers at The University of Texas at Austin wanted to understand how the brain processes language. So Alexander Huth, a brain and computer scientist, says they had three people spend up to 16 hours in an fMRI scanner, listening to podcasts.
ALEXANDER HUTH: For the most part, they just lay there and listen to mostly stories from "The Moth Radio Hour."
HAMILTON: Huth says those personal stories produced activity all over the brain, not just in areas associated with speech and language.
HUTH: It turns out that a huge amount of the brain is doing something - so areas that we use for navigation, areas that we use for doing mental math, areas that we use for processing what things feel like to touch.
HAMILTON: The team had a computer learn to match specific patterns of brain activity with certain streams of words. Then, participants listened to some news stories, and the computer tried to reconstruct these stories from each person's brain activity. It got help from an early version of the natural language processing program, ChatGPT. Huth says the technology is far from perfect, but usually gets the gist of a story right.
HUTH: The real story said, I didn't even have my driver's license yet, and then the decoder version is, she hadn't even learned to drive yet.
HAMILTON: Huth says the system tends to mess up pronouns and rarely repeats the exact words a person was hearing or thinking.
HUTH: It's getting at - what are the ideas behind the words, the semantics, the meaning?
HAMILTON: The team confirmed that with a different experiment. They had the system decode participants' brain activity as they watched wordless videos.
HUTH: We didn't tell the subjects to, like, try to describe in your head what's happening. Just watch the video. Just enjoy the video. And yet what we got out was this kind of language description of what's going on in the video.
HAMILTON: The approach, described in the journal "Nature Neuroscience," might someday help people who are paralyzed regain the ability to communicate. But Huth says this sort of technology might also be abused.
HUTH: This is very exciting, but it's also, like, a little scary. There are implications here in terms of, like, mental privacy, right? What if you can read out the words that somebody is just thinking in their head? That's potentially a harmful thing.
HAMILTON: That's not possible now. Decoding only works when a participant is actively trying to help. But David Moses, a brain scientist at the University of California San Francisco, says that might change someday.
DAVID MOSES: This is all about the user having a new way of communicating, a new tool that is totally in their control. That's the goal. And we have to make sure that that stays the goal.
HAMILTON: Moses is part of a lab that's been using a different approach to detecting words in the brain.
MOSES: People get a sheet of electrical sensors implanted directly on the surface of the brain, and that records brain activity really close to the source.
HAMILTON: The implanted sensors are faster and more accurate. But Moses says there's one clear advantage with MRI.
MOSES: This is noninvasive. No one has to get brain surgery to use an MRI machine.
HAMILTON: Communication may be just one application for MRI-based language decoding. Marcel Just of Carnegie Mellon University says the technology could help scientists understand how conditions like suicidal depression or schizophrenia alter the brain.
MARCEL JUST: One of the biggest scientific medical challenges is understanding mental illness, which is a brain dysfunction, ultimately. I think that this general kind of approach is going to solve that puzzle someday.
HAMILTON: Perhaps by going beyond mere words to reveal the stream of thoughts and ideas in a person's head. Jon Hamilton, NPR News.
(SOUNDBITE OF MINUTEMEN'S "COHESION")
Copyright © 2023 NPR. All rights reserved. Visit our website terms of use and permissions pages at www.npr.org for further information.
NPR transcripts are created on a rush deadline by an NPR contractor. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.