There were jokes, but the comedy host also ventured into serious topics. Letterman asked, "And how is the economy?"
"It is improving," Obama said. "We have seen some stabilizing. The financial markets aren't in meltdown. You've actually seen an uptick in investment, and even manufacturing, which had been really getting battered, has started to improve. But we are not out of the woods yet."
After one commercial break, Letterman asked about the shouting and anger at town halls this summer, and about how some critics have vilified the president. He noted that former President Jimmy Carter said some of it is rooted is racism.
"Is he onto something there, or is that just something to talk about?" Letterman asked.
"First of all," Obama began his answer, "I think it's important to realize that I was actually black before the election."
The audience roared with laughter.
In the show's final minutes, Letterman asked about Iraq. The president said now the country has a chance to establish a democracy, and the removal of U.S. troops is on track.
But on Afghanistan, Obama said it's a difficult mission that he is reassessing: "Before I send some young man and woman in uniform over there, and I'm answerable to their parents — and if they don't come back, I've got to write a letter to them saying that their child has sacrificed on behalf of America — before I make those decisions, I've got to make sure that the policy in place is worthy of their sacrifice."
The response garnered applause from the audience.
The president will address the opening session of the U.N. General Assembly on Wednesday.
But to get his message out to Americans, Obama recognized that a visit with a late-night comic is also time well spent.