Copyright ©2009 NPR. For personal, noncommercial use only. See Terms of Use. For other uses, prior permission required.


Gil Weinberg plays jazz, so does Shimon. Weinberg is a Georgia Tech computer scientist; Shimon is a robotic marimba player. Here's a trick question: which one do you think can improvise on command in the style of Thelonious Monk or John Coltrane? Well, Weinberg says his creation, Shimon, can. He says the robot can listen to what's played, analyze it and then improvise. And Shimon has been programmed for different styles of improvisation.

Weinberg spoke to us from Japan, where over the weekend, participants in what was billed as an intercontinental human-robot interaction played a few notes using their mobile phones, and Shimon listened and joined in from Atlanta. Weinberg says Shimon isn't just responsive; he is creative.

Dr. GIL WEINBERG (Director of Music Technology, Georgia Institute of Technology): The whole idea is to use computer algorithms to create music in ways that humans will never create. Our motto is: Listen like a human, but improvise like a machine.

SIEGEL: Now, you have programmed Shimon to do something which is both extremely intriguing and, I would think, very challenging, which is to play like Thelonious Monk.

Dr. WEINBERG: Right. For that, we tried to have a robot that can play like a machine. But first, we want it to try to play like a human. So what we use is statistics. We actually ran a lot of analysis on Thelonious Monk's improvisation. And once you get a statistical model of Thelonious Monk, you can improvise in this model. So in the last piece that we're currently playing in Japan, humans in Japan play the application that allows you to create musical feeds.

(Soundbite of music)

Dr. WEINBERG: And he listens to this feed and improvises based on Thelonious Monk's style.

(Soundbite of music)

Dr. WEINBERG: Or John Coltrane.

(Soundbite of music)

Dr. WEINBERG: And the player here in Japan can actually change the style using this iPhone application.

(Soundbite of music)

Dr. WEINBERG: So if, for example, you play something and you want to hear it like Thelonious Monk would play it, you can change a slider on the iPhone. Then if you want to insert more of your own idea, you can change the slider and make your own original idea more prominent in the improvisation. And you can also play along by shaking the phone.

SIEGEL: I want you to deal with the logical concept of predicting improvisation, predicting what would be the likely improvisation. Can one analyze what Coltrane would do, what Monk would do, with such predictability that we can find what they would likely do with something?

Dr. WEINBERG: It's difficult to predict exactly what they would do in every single moment in time. But our algorithm pretty much looks at the past several notes that it plays. And based on that, it sees what is the probability of the next note to be based on all of this analysis of a large corpus of transcribed improvisation.

It will not sound exactly like the original jazz master will play like, but it probably will keep the nature and the character of its style.

SIEGEL: Are there, by the way, say, improvisers who are just beyond the reach of computers? Is Ornette Coleman less reducible to algorithm than a good bebop jazz musician like Monk?

Dr. WEINBERG: For Ornette Coleman, we'll probably have to have a very big corpus of improvisation to really try to analyze enough information that will give us a good idea of how Coleman played.

In a sense, it kind of reduces music to numbers and statistics. One can say is it just any computer program can reproduce the genius of these masters. But I think that with tweaking enough the algorithms that we'll be able to create something that will be very similar to the jazz master.

I don't think that the robot should try to play like a human. I think the human's doing a great job in what humans are great at. In all the emotional and expressive energy, I don't think a robot can capture.

SIEGEL: Why can't you imagine quantifying expression and emotion? Why couldn't you feed enough music into a computer to figure out what different feelings are and cue feelings through combinations of other sides and, in fact, quantify that which we consider to be the unquantifiable human dimension of music?

Dr. WEINBERG: I don't think we have the math for that yet. We have some math to get the notes and the rhythms and the scales. Whether this can capture the genius of Thelonious Monk, I, you know, I hope not. But maybe.

SIEGEL: Well, Gil Weinberg, thank you very much for talking with us about it.

Dr. WEINBERG: I appreciate it.

SIEGEL: And for introducing us to Shimon.

Dr. WEINBERG: Thanks so much.

SIEGEL: That was Gil Weinberg of Georgia Tech, speaking to us from Japan. And you can see Shimon getting down on the marimba at the ALL TECH CONSIDERED blog:

(Soundbite of music)


You're listening to ALL THINGS CONSIDERED from NPR News.

Copyright © 2009 NPR. All rights reserved. No quotes from the materials contained herein may be used in any media without attribution to NPR. This transcript is provided for personal, noncommercial use only, pursuant to our Terms of Use. Any other use requires NPR's prior permission. Visit our permissions page for further information.

NPR transcripts are created on a rush deadline by a contractor for NPR, and accuracy and availability may vary. This text may not be in its final form and may be updated or revised in the future. Please be aware that the authoritative record of NPR's programming is the audio.



Please keep your community civil. All comments must follow the Community rules and Terms of Use. NPR reserves the right to use the comments we receive, in whole or in part, and to use the commenter's name and location, in any medium. See also the Terms of Use, Privacy Policy and Community FAQ.