New AI voice-cloning tools ‘add fuel’ to misinformation fire

Feb 10, 2023, 4:27 PM | Updated: 4:30 pm

NEW YORK (AP) — In a video from a Jan. 25 news report, President Joe Biden talks about tanks. But a doctored version of the video has amassed hundred of thousands of views this week on social media, making it appear he gave a speech that attacks transgender people.

Digital forensics experts say the video was created using a new generation of artificial intelligence tools, which allow anyone to quickly generate audio simulating a person’s voice with a few clicks of a button. And while the Biden clip on social media may have failed to fool most users this time, the clip shows how easy it now is for people to generate hateful and disinformation-filled “deepfake” videos that could do real-world harm.

“Tools like this are going to basically add more fuel to fire,” said Hafiz Malik, a professor of electrical and computer engineering at the University of Michigan who focuses on multimedia forensics. “The monster is already on the loose.”

It arrived last month with the beta phase of ElevenLabs’ voice synthesis platform, which allowed users to generate realistic audio of any person’s voice by uploading a few minutes of audio samples and typing in any text for it to say.

The startup says the technology was developed to dub audio in different languages for movies, audiobooks and gaming to preserve the speaker’s voice and emotions.

Social media users quickly began sharing an AI-generated audio sample of Hillary Clinton reading the same transphobic text featured in the Biden clip, along with fake audio clips of Bill Gates supposedly saying that the COVID-19 vaccine causes AIDS and actress Emma Watson purportedly reading Hitler’s manifesto “Mein Kampf.”

Shortly after, ElevenLabs tweeted that it was seeing “an increasing number of voice cloning misuse cases,” and announced that it was now exploring safeguards to tamp down on abuse. One of the first steps was to make the feature available only to those who provide payment information. Initially, anonymous users were able to access the voice cloning tool for free. The company also claims that if there are issues, it can trace any generated audio back to the creator.

But even the ability to track creators won’t mitigate the tool’s harm, said Hany Farid, a professor at the University of California, Berkeley, who focuses on digital forensics and misinformation.

“The damage is done,” he said.

As an example, Farid said bad actors could move the stock market with fake audio of a top CEO saying profits are down. And already there’s a clip on YouTube that used the tool to alter a video to make it appear Biden said the U.S. was launching a nuclear attack against Russia.

Free and open-source software with the same capabilities have also emerged online, meaning paywalls on commercial tools aren’t an impediment. Using one free online model, the AP generated audio samples to sound like actors Daniel Craig and Jennifer Lawrence in just a few minutes.

“The question is where to point the finger and how to put the genie back in the bottle?” Malik said. “We can’t do it.”

When deepfakes first made headlines about five years ago, they were easy enough to detect since the subject didn’t blink and audio sounded robotic. That’s no longer the case as the tools become more sophisticated.

The altered video of Biden making derogatory comments about transgender people, for instance, combined the AI-generated audio with a real clip of the president, taken from a Jan. 25 CNN live broadcast announcing the U.S. dispatch of tanks to Ukraine. Biden’s mouth was manipulated in the video to match the audio. While most Twitter users recognized that the content was not something Biden was likely to say, they were nevertheless shocked at how realistic it appeared. Others appeared to believe it was real – or at least didn’t know what to believe.

Hollywood studios have long been able to distort reality, but access to that technology has been democratized without considering the implications, said Farid.

“It’s a combination of the very, very powerful AI based technology, the ease of use, and then the fact that the model seems to be: let’s put it on the internet and see what happens next,” Farid said.

Audio is just one area where AI-generated misinformation poses a threat.

Free online AI image generators like Midjourney and DALL-E can churn out photorealistic images of war and natural disasters in the style of legacy media outlets with a simple text prompt. Last month, some school districts in the U.S. began blocking ChatGPT, which can produce readable text – like student term papers – on demand.

ElevenLabs did not respond to a request for comment.

Copyright © The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.

AP

(Mike Stocker/South Florida Sun-Sentinel via AP)Credit: ASSOCIATED PRESS...

Associated Press

Florida police search for 3 gunmen who wounded 9 at crowded beach on Memorial Day

Police are responding to a shooting near the beach broadwalk in Hollywood, Florida.

2 days ago

Crew members assemble the main stage ahead of the 2023 Scripps Nations Spelling Bee on Sunday, May ...

Associated Press

Exclusive secrets of the National Spelling Bee: Picking the words to identify a champion

As the final pre-competition meeting of the Scripps National Spelling Bee's word selection panel stretches into its seventh hour, the pronouncers no longer seem to care.

2 days ago

FILE - Gabby Petito's mother Nichole Schmidt, wipes a tear from her face during a news conference o...

Associated Press

Mother of man who killed Gabby Petito said in letter she would help son ‘dispose of a body’

The mother of the man who killed Gabby Petito told her son in an undated letter that she would “dispose of a body” if needed because she loved him so much, according to copies of the note shared publicly for the first time this week by attorneys for Petito's parents.

5 days ago

A member of the 3rd U.S. Infantry Regiment, also known as The Old Guard, places flags in front of e...

Associated Press

5 things to know about Memorial Day including its controversies

Memorial Day is supposed to be about mourning the nation’s fallen service members, but it’s come to anchor the unofficial start of summer and a long weekend of discounts on anything from mattresses to lawn mowers.

5 days ago

FILE - This artist sketch depicts the trial of Oath Keepers leader Stewart Rhodes, left, as he test...

Associated Press

Officers describe chaos, fear on Jan. 6 as judge weighs prison time for Oath Keepers’ Rhodes

Police officers who defended the U.S. Capitol on Jan. 6, 2021, and public servants who fled the mob's attack told a judge on Wednesday that they are still haunted by what they endured, as the judge prepares to hand down sentences in a landmark Capitol riot case.

6 days ago

Pride month merchandise is displayed at the front of a Target store in Hackensack, N.J., Wednesday,...

Associated Press

Target on the defensive after removing LGBTQ+-themed products

Target once distinguished itself as being boldly supportive of the LGBTQ+ community.

7 days ago

Sponsored Articles

...

re:vitalize

Why drug-free weight loss still matters

Wanting to lose weight is a common goal for many people as they progress throughout life, but choosing between a holistic approach or to take medicine can be a tough decision.

...

OCD & Anxiety Treatment Center

How to identify the symptoms of 3 common anxiety disorders

Living with an anxiety disorder can be debilitating and cause significant stress for those who suffer from the condition.

...

Day & Night Air Conditioning, Heating and Plumbing

Company looking for oldest air conditioner and wants to reward homeowner with new one

Does your air conditioner make weird noises or a burning smell when it starts? If so, you may be due for an AC unit replacement.

New AI voice-cloning tools ‘add fuel’ to misinformation fire