How Twitter's Trending Algorithm Picks Its Topics

Occupy Wall Street protesters meditate while a sign bearing their Twitter hashtag hangs from a railing in Zuccotti Park in October. Some activists accused Twitter of censorship because #OccupyWallStreet wasn't appearing on trending lists.

Occupy Wall Street protesters meditate while a sign bearing their Twitter hashtag hangs from a railing in Zuccotti Park in October. Some activists accused Twitter of censorship because #OccupyWallStreet wasn't appearing on trending lists. Jessica Rinaldi/Reuters/Landov hide caption

itoggle caption Jessica Rinaldi/Reuters/Landov

The list of "trending topics" on the right side of Twitter's home page is a coveted spot because millions of people see it. It often reflects what's hot in the news, from the death of Steve Jobs to Kim Kardashian's latest exploits.

Sometimes a topic that seems hot, like Occupy Wall Street, doesn't trend, leading some activists to charge Twitter with censorship. But the complex algorithms that determine trending topics are intended to find what's trending in the moment, and not what's been around for a long time.

Getting a spot on the trending list has become so important that television programs hire consultants to help them get there.

Jason Pollock, a social media consultant, says the singing competition The Voice set the tone for Twitter interaction. On the show, teams of young singers get coached by stars like Christina Aguilera. The Voice became one of the most popular singing competitions on TV this year with the help of Twitter.

Pollock says that as competitors are training for the competition, they tweet with some help from paid staff. They put pound signs in front of catchy phrases to create hashtags, like #TeamNakia. "They had them really operating fast. ... They had hashtags going and they were asking questions and doing polls and really engaging stuff," Pollock says.

When those hashtags hit the trending topics list on Twitter, more people started to watch The Voice on TV.

"If you start trending on Twitter, people on Twitter will turn that show on," Pollock explains. "Everyone is obsessed about the ratings — well, here's a new tool to up your ratings."

Hashtags Over Time

Social media company SocialFlow looked at the entire Twitter output over the course of a few weeks to see when different hashtags became popular. The #ThankYouSteve spike occurred in response to Steve Jobs' death; other hashtags responded to TV specials or memes that became popular. The #OccupyWallStreet hashtag, in red, has a more enduring popularity over time than the spikes.

 

#OccupyWallStreet Not On The List

The trending list can also bring attention to your cause. During the tense moments of Occupy Oakland, freelance journalist Susie Cagle was following the Occupy Wall Street hashtag.

"Just looking at how often I was tweeting, let alone how often others were on the same Occupy Oakland hashtag, there have been times where that hashtag is scrolling and updating so quickly that there's no way I can keep up," she says.

But it struck her as odd that Occupy Oakland wasn't on trending topics, even after weeks of protests.

"To not see those things trending a month in was surprising," she says.

On blogs, some Occupy Wall Street activists charged Twitter with censoring its trending topics list. Why were there trending hashtags like #WhatYouFindInLadiesHandbags or #ThingsThirstyPeopleDo?

Twitter's Matt Graves says it's because the protests had been going on for weeks.

"We look at trending topics as a reflection of what people are talking about more right now in this moment, than they were a minute ago, an hour ago or a day ago," Graves says.

Where Do Trends Come From?

Twitter's trending algorithm finds topics that many people suddenly tweet about at once, like new album releases, TV shows on air, celebrity deaths or political gaffes. The hot topics can also appear when many people rush to Twitter to talk about one thing, like an earthquake. But if a lot of people talk about rain, it probably won't make the list since it's a more common occurrence.

An HP Labs study found that 31 percent of trending topics come from retweets; 72 percent of those originate from around 20 major news sources mostly based in the U.S., UK and Brazil. An outlet like CNN or the BBC might tweet about a breaking story, which gets heavily retweeted until it appears in the trending list where other people continue to talk about it.

Hashtags also frequently trend if they're new and haven't been through the algorithm before. Topics can start trending, fall off the list and later reappear when a different set of users discovers the topic later on. They usually only appear for around 40 minutes at a time, but the trending list is such a prime spot for marketers that a promoted space at the top runs $120,000 per day, according to Twitter's director of revenue.

Sara Carothers

Source: HP Social Computing Lab

And when Graves says "we," he's really talking about Twitter's secret algorithm that sorts through 250 million tweets a day. It's designed to search for the sudden appearance of a topic in large volume.

"That may mean, for example, when Steve Jobs [died], there are different trending topics around that," Graves says. "It may mean on a local level, when there's an earthquake — whether it's in Oklahoma or California — being able to go and see, well, is 'earthquake' trending in Oklahoma right now?"

Getting Used To An Algorithmic Editor

There had been thousands of tweets for Occupy Wall Street regularly over many weeks, so Twitter's algorithms stopped putting it on the trending topics list. In some ways, Twitter's algorithms act like a lot of human news editors who are more interested in the latest news than an ongoing story, says Tarleton Gillespie, a communications professor at Cornell University.

"It's going to be hard for us to think about the involvement of the algorithm in a way that we could understand the involvement of an editor ... [who] might step in and say, 'I don't want to run this story' or 'I think this one's more important than that one,' " Gillespie says. "What does it mean that an algorithm decided that something was more important than something else?"

Gillespie points to the Amazon best-seller list. In that case, it seems like a simple unbiased calculation.

"Naively, you would say, 'Well, the most-selling book is No. 1 and the second-selling book is No. 2,' " he says.

But a couple of years ago, there was a dust-up because all the gay-themed books disappeared from the list. It turns out that Amazon doesn't let any adult books in its best-sellers, and someone accidentally put the gay-themed books in that category.

"It's a curated list," Gillespie says. "It's a list that will never show us if something that they or their publishers had classified as adult would ever show up there."

Everything from restaurant reviews to your friend's baby pictures to your local news is getting served up to you by an algorithm. As much as programmers may think their algorithms will deliver objective results, those calculations may be just as biased as a real human being.

Comments

 

Please keep your community civil. All comments must follow the NPR.org Community rules and terms of use, and will be moderated prior to posting. NPR reserves the right to use the comments we receive, in whole or in part, and to use the commenter's name and location, in any medium. See also the Terms of Use, Privacy Policy and Community FAQ.