Insider Q&A: Trust and safety exec talks about AI and content moderation

Apr 22, 2024, 1:04 PM

Alex Popken was a longtime trust and safety executive at Twitter focusing on content moderation before leaving in 2023. She was the first employee there dedicated to moderating Twitter’s advertising business when she started in 2013.

Now, she’s vice president of trust and safety at WebPurify, a content moderation service provider that works with businesses to help ensure the content people post on their sites follows the rules.

Social media platforms are not the only ones that need policing. Any consumer-facing company — from retailers to dating apps to news sites — needs someone to weed out unwanted content, whether that’s hate speech, harassment or anything illegal. Companies are increasingly using artificial intelligence in their efforts, but Popken notes that humans remain essential to the process.

Popken spoke recently with The Associated Press. The conversation has been edited for clarity and length.

QUESTION: How did you see content moderation change in that decade you were at Twitter?

ANSWER: When I joined Twitter, content moderation was in its nascent stages. I think even trust and safety was this concept that people were just starting to understand and grapple with. The need for content moderation escalated as we, as platforms saw them be weaponized in new ways. I can sort of recall some key milestones of my tenure at Twitter. For example, Russian interference in the 2016 U.S. presidential election, where we realized for the first time, realized in a meaningful way, that without content moderation we can have bad actors undermining democracy. The necessity for investing in this area became ever more important.

Q: A lot of companies, the bigger social media companies are leaning on AI for content moderation. Do you think that AI is in a place yet where it’s possible to rely on it?

A: Effective content moderation is a combination of humans and machines. AI, which has been used in moderation for years, solves for scale. And so you have machine learning models that are trained on different policies and can detect content. But ultimately, let’s say you have a machine learning model that is detecting the word ‘Nazi.’ There are a lot of posts that might be criticizing Nazis or providing educational material about Nazis versus like, white supremacy. And so it cannot solve for nuance and context. And that’s really where a human layer comes in.

I do think that we’re starting to see really important advancements that are going to make a human’s job easier. And I think generative AI is a great example of that, where, unlike traditional. AI models, it can understand context and nuance much more so than its predecessor. But even still, we have entirely new use cases for our human moderators now around moderating generative AI outputs. And so the need for human moderation will remain for the foreseeable future, in my opinion.

Q: Can you talk a little bit about the non-social media companies that you work with and what kind of content moderation they use?

A: I mean, everything from like retail product customization, you know, imagine that you are allowing people to customize T-shirts, right? Obviously, you want to avoid use cases in which people abuse that and put harmful, hateful things on the T-shirt.

Really, anything that has user-generated content, all the way to online dating — there, you’re looking for things like catfishing and scams and ensuring that people are who they say they are and preventing people from uploading inappropriate photos for example. It does span multiple industries.

Q: What about the issues that you’re moderating, does that change?

A: Content moderation is an ever-evolving landscape. And it’s influenced by what’s happening in the world. It’s influenced by new and evolving technologies. It’s influenced by bad actors who will attempt to get on these platforms in new and innovative ways. And so as a content moderation team, you’re trying to stay one step ahead and anticipate new risks.

I think that there’s a little bit of catastrophic thinking in this role where you think about like, what are the worst case scenarios that can happen here. And certainly they evolve. I think misinformation is a great example where there’s so many facets to misinformation and it’s such a hard thing to moderate. It’s like boiling the ocean. I mean, you cannot fact check every single thing that someone says, right? And so typically platforms need to focus on misinformation not to cause the most real world harm. And that’s also always evolving.

Q: In terms of generative AI there’s some doomsday thinking that it will ruin the internet, that it will just be, you know, fake AI stuff on it. Do you feel like that might be happening?

A: I have concerns around AI-generated misinformation, especially during what is an extremely important election season globally. You know, we actively are seeing more deepfakes and harmful synthetic and manipulated media online, which is concerning because I think the average person probably has a hard time. discerning accurate versus not.

I think medium to long term, if I can be properly regulated and if there are appropriate guardrails around it, I also think that it can create an opportunity for our trust and safety practitioners. I do. Imagine a world in which AI is an important tool in the tool belt of content moderation, for things like threat intelligence. You know, I think that it’s going to be extremely helpful tool, but it’s also going to be misused. And we’re we’re already seeing that.

United States News

Associated Press

Three little piggies at a yoga class = maximum happiness

SPENCER, Mass. (AP) — Three little piggies went to a yoga class. Their human companions had a blast. Wilbur, Charlotte and Bluey fit right into a growing trend of yoga with animals, adding some fun to the usual physical and mental wellness exercises at a class in central Massachusetts. Darting and strolling among the yoginis […]

5 hours ago

Associated Press

Remember last year’s Memorial Day travel jams? Chances are they will be much worse this year

You didn’t think summer travel would be easy, did you? Highways and airports are likely to be jammed the next few days as Americans head out for Memorial Day weekend getaways and then return home. AAA predicts this will be the busiest start-of-summer weekend in nearly 20 years, with 43.8 million people expected to travel […]

5 hours ago

Associated Press

5 dead and nearly 3 dozen hurt in tornadoes that tore through Iowa, officials say

GREENFIELD, Iowa (AP) — A deadly tornado that wreaked havoc in the small city of Greenfield, Iowa, left four people dead and nearly three dozen injured, officials said, while a fifth person was killed elsewhere. The twister that tore through the city on Tuesday was rated at least an EF-3 by the National Weather Service […]

6 hours ago

Associated Press

Thousands of journalists have fled homelands due to repression, threats and conflict, UN expert says

UNITED NATIONS (AP) — Thousands of journalists have fled their home countries in recent years to escape political repression, save their lives and escape conflict – but in exile they are often vulnerable to physical, digital and legal threats, a U.N. investigator said Wednesday. Irene Khan said in a report to the U.N. General Assembly […]

6 hours ago

Javier Cazares, center, stands with families of the victims in the Uvalde elementary school shootin...

Associated Press

Families of Uvalde school shooting victims file $500 million federal lawsuit against Texas state police

The families of the victims in the Uvalde school shooting in Texas filed a $500 million federal lawsuit against 92 state police officers.

8 hours ago

Javier Cazares, center, stands with families of the victims in the Uvalde elementary school shootin...

Sponsored Content by

The families of the victims in the Uvalde school shooting in Texas filed a $500 million federal lawsuit against 92 state police officers.

Sponsored Articles


Collins Comfort Masters

Here’s how to be worry-free when your A/C goes out in the middle of summer

PHOENIX -- As Arizona approaches another hot summer, Phoenix residents are likely to spend more time indoors.


Day & Night Air Conditioning, Heating and Plumbing

Beat the heat, ensure your AC unit is summer-ready

With temperatures starting to rise across the Valley, now is a great time to be sure your AC unit is ready to withstand the sweltering summer heat.


Midwestern University

Midwestern University Clinics: transforming health care in the valley

Midwestern University, long a fixture of comprehensive health care education in the West Valley, is also a recognized leader in community health care.

Insider Q&A: Trust and safety exec talks about AI and content moderation