NOEL KING, HOST:
TikTok at its best is the joke dance videos. We all know that. But it is not all fun stuff. The platform is confronting a surge of misinformation and extremist content. So TikTok is learning from Facebook and Twitter, which have been dealing with this for a while. Here's NPR's Bobby Allyn.
BOBBY ALLYN, BYLINE: Early on in the pandemic, a guy in Brooklyn walked by a hospital and took out his phone. He opened TikTok and started filming. It was part of a disinformation campaign to frame the coronavirus as a hoax.
(SOUNDBITE OF TIKTOK VIDEO)
UNIDENTIFIED PERSON: Not much happening. And this is a hospital that serves thousands and thousands of people here in downtown Brooklyn, N.Y. There is no mass chaos out here, contrary to what the mainstream media is telling you.
ALLYN: He went by the username saynotosocialism on TikTok. Now, of course, believing that recording the outside of a hospital proves anything at all is absurd. But the Trump-supporting conspiracy theory QAnon endorsed the idea. And on TikTok, it caught on.
ANGELO CARUSONE: More people saw it. More people were going to film their hospital, which in turn was getting them video views and incentivizing more people to do it. So it was this incredible feedback loop.
ALLYN: That's Angelo Carusone, president of the left-leaning watchdog group Media Matters, which found more than a dozen QAnon hashtags on TikTok that together garnered hundreds of millions of views. TikTok has since banned all QAnon content. But Carusone says TikTok's algorithm rewards engagement. And hoaxes - they could be really engaging.
CARUSONE: The algorithm is sort of like the recommendation engines of YouTube but on steroids.
ALLYN: The power and reach of the app isn't lost on other fringe groups. Earlier this month, Brandon Caserta was arrested in connection with a plot to kidnap the governor of Michigan. Before that happened, though, he posted this TikTok, previewing his, quote, "recon plan."
(SOUNDBITE OF TIKTOK VIDEO)
BRANDON CASERTA: Guess what? I'm sick of being robbed and enslaved by the state, period. I'm sick of it. And these are the guys who are actually doing it.
ALLYN: TikTok took down Caserta's video and the earlier one of the hospital. TikTok says it will remove all accounts sharing hate and misinformation. But in reality, some extremists have learned how to outfox the app's defenses - both human and AI. The Anti-Defamation League's Dave Sifry found that white supremacists are using code language on TikTok to find new recruits.
DAVE SIFRY: The number one for the letter I or L. And so you might see n-4-z-l standing for Nazi.
ALLYN: Another common tactic among extremists on TikTok is attempting to hijack a trending topic.
SIFRY: They might use the hashtag #BlackLivesMatter as a way to actually promulgate some of their hateful ideologies to people who are searching for other kinds of videos.
ALLYN: But those who work with TikTok say it deserves credit for not repeating the mistakes of older social networks like Facebook and Twitter. TikTok has learned that a scattershot method of just taking down bad stuff as it appears doesn't work. The controls have to be proactive. Hany Farid is a UC Berkeley computer science professor who sits on a TikTok advisory panel.
HANY FARID: Ah, here's a piece of content. Ah, here's a group. Let's ban this. Let's not ban that. And there's no coherent policy. And I think TikTok has the advantage of coming to this game relatively late and seeing what has not worked with the other social media companies.
ALLYN: Unlike other social networks, Farid says TikTok sees itself as entertainment. And that's mostly what you see on the platform - dances, jokes, pranks, ridiculous dog videos. Hateful and violent videos are made, too, though TikTok is good at catching it. Still, Farid admits it's there if you look hard enough.
FARID: And I'm not excusing it. I'm not saying, well, you know, that's the price you pay. But, you know, just playing defense is hard. And no matter how good the algorithms get, no matter how many moderators you hire, there's going to be mistakes.
ALLYN: TikTok has twice as many U.S. users as Twitter. Those are a lot of people making a lot of videos. Mistakes are just an unavoidable result of trying to police them all. Bobby Allyn, NPR News, San Francisco.
(SOUNDBITE OF PENSEES'S "LAGUNA")
NPR transcripts are created on a rush deadline by Verb8tm, Inc., an NPR contractor, and produced using a proprietary transcription process developed with NPR. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.