UNITED STATES NEWS

Hackers aim to find flaws in AI – with White House help

May 9, 2023, 4:15 PM

No sooner did ChatGPT get unleashed than hackers started “jailbreaking” the artificial intelligence chatbot – trying to override its safeguards so it could blurt out something unhinged or obscene.

But now its maker, OpenAI, and other major AI providers such as Google and Microsoft, are coordinating with the Biden administration to let thousands of hackers take a shot at testing the limits of their technology.

Some of the things they’ll be looking to find: How can chatbots be manipulated to cause harm? Will they share the private information we confide in them to other users? And why do they assume a doctor is a man and a nurse is a woman?

“This is why we need thousands of people,” said Rumman Chowdhury, lead coordinator of the mass hacking event planned for this summer’s DEF CON hacker convention in Las Vegas that’s expected to draw several thousand people. “We need a lot of people with a wide range of lived experiences, subject matter expertise and backgrounds hacking at these models and trying to find problems that can then go be fixed.”

Anyone who’s tried ChatGPT, Microsoft’s Bing chatbot or Google’s Bard will have quickly learned that they have a tendency built on what’s known as large language models, also emulate the cultural biases they’ve learned from being trained upon huge troves of what people have written online.

The idea of a mass hack caught the attention of U.S. government officials in March at the South by Southwest festival in Austin, Texas, where Sven Cattell, founder of DEF CON’s long-running AI Village, and Austin Carson, president of responsible AI nonprofit SeedAI, helped lead a workshop inviting community college students to hack an AI model.

Carson said those conversations eventually blossomed into a proposal to test AI language models following the guidelines of give users control over their data and ensure that automated systems are used safely and transparently.

There’s already a community of users trying their best to trick chatbots and highlight their flaws. Some are official “red teams” authorized by the companies to “prompt attack” the AI models to discover their vulnerabilities. Many others are hobbyists showing off humorous or disturbing outputs on social media until they get banned for violating a product’s terms of service.

“What happens now is kind of a scattershot approach where people find stuff, it goes viral on Twitter,” and then it may or may not get fixed if it’s egregious enough or the person calling attention to it is influential, Chowdhury said.

In one example, known as the “grandma exploit,” users were able to get chatbots to tell them how to make a bomb — a request a commercial chatbot would normally decline — by asking it to pretend it was a grandmother telling a bedtime story about how to make a bomb.

In another example, searching for Chowdhury using an early version of Microsoft’s Bing search engine chatbot — which is based on the same technology as ChatGPT but can pull real-time information from the internet — led to a profile that speculated Chowdhury “loves to buy new shoes every month” and made strange and gendered assertions about her physical appearance.

Chowdhury helped introduce a method for rewarding the discovery of algorithmic bias to DEF CON’s AI Village in 2021 when she was the head of Twitter’s AI ethics team — a job that has since been eliminated upon Elon Musk’s October takeover of the company. Paying hackers a “bounty” if they uncover a security bug is commonplace in the cybersecurity industry — but it was a newer concept to researchers studying harmful AI bias.

This year’s event will be at a much greater scale, and is the first to tackle the large language models that have attracted a surge of public interest and commercial investment since the release of ChatGPT late last year.

Chowdhury, now the co-founder of AI accountability nonprofit Humane Intelligence, said it’s not just about finding flaws but about figuring out ways to fix them.

“This is a direct pipeline to give feedback to companies,” she said. “It’s not like we’re just doing this hackathon and everybody’s going home. We’re going to be spending months after the exercise compiling a report, explaining common vulnerabilities, things that came up, patterns we saw.”

Some of the details are still being negotiated, but companies that have agreed to provide their models for testing include OpenAI, Google, chipmaker Nvidia and startups Anthropic, Hugging Face and Stability AI. Building the platform for the testing is another startup called Scale AI, known for its work in assigning humans to help train AI models by labeling data.

“As these foundation models become more and more widespread, it’s really critical that we do everything we can to ensure their safety,” said Scale CEO Alexandr Wang. “You can imagine somebody on one side of the world asking it some very sensitive or detailed questions, including some of their personal information. You don’t want any of that information leaking to any other user.”

Other dangers Wang worries about are chatbots that give out “unbelievably bad medical advice” or other misinformation that can cause serious harm.

Anthropic co-founder Jack Clark said the DEF CON event will hopefully be the start of a deeper commitment from AI developers to measure and evaluate the safety of the systems they are building.

“Our basic view is that AI systems will need third-party assessments, both before deployment and after deployment. Red-teaming is one way that you can do that,” Clark said. “We need to get practice at figuring out how to do this. It hasn’t really been done before.”

United States News

This photo provided by Las Vegas Metropolitan Police Department shows a monolith near Gass Peak, Ne...

Associated Press

Gleaming monolith pops up in Nevada desert, the latest in a series of quickly vanishing structures

Jutting out of the rocks in a remote mountain range near Las Vegas, the monolith's reflective surface imitates the vast desert landscape surrounding the mountain peak where it has been erected.

1 hour ago

Associated Press

California man charged with killing gay college student takes the stand

SANTA ANA, Calif. (AP) — A California man charged with killing a gay University of Pennsylvania student in an alleged hate crime six years ago testified Tuesday about how the two men connected on social media and went to a park. It was the last time the victim was seen. Samuel Woodward, 26, took the […]

3 hours ago

President Joe Biden turns to the others on the dais after speaking during an event marking the 12th...

Associated Press

Half a million immigrants could eventually get US citizenship under a sweeping new plan from Biden

President Joe Biden ordered expansive election-year action Tuesday to offer potential citizenship to hundreds of thousands of immigrants without legal status in the U.S.

3 hours ago

Associated Press

Alaska did not provide accessible voting for those with disabilities, US Justice Department alleges

ANCHORAGE, Alaska (AP) — The state of Alaska has violated the Americans with Disabilities Act for not providing accessible machines for in-person voting, the U.S. Department of Justice said Tuesday. The state was also faulted for selecting inaccessible polling places and operating a state elections website that can’t be accessed by everyone. The department informed […]

3 hours ago

Associated Press

New York’s ‘equal rights’ constitutional amendment restored to ballot by appeals court

ALBANY, N.Y. (AP) — A proposed amendment to New York’s constitution barring discrimination based on “gender identity” and “pregnancy outcomes” was restored to the November election ballot Tuesday by a state appeals court. In a short decision, a panel of midlevel appellate judges overturned a May decision by an upstate judge to strike the proposed […]

4 hours ago

Associated Press

Texas megachurch pastor resigns after woman says he sexually abused her in the 1980s

DALLAS (AP) — The pastor of a Texas megachurch has resigned after a woman said he had sexually abused her on multiple occasions in the 1980s, beginning when she was 12. Gateway Church’s board of elders said in a statement Tuesday that they’d accepted the resignation of Robert Morris, the church’s senior pastor and founder. […]

5 hours ago

Sponsored Articles

...

Sanderson Ford

3 new rides for 3 new road trips in Arizona

It's time for the Sanderson Ford Memorial Day sale with the Mighty Fine 69 Anniversary, as Sanderson Ford turned 69 years old in May.

...

DISC Desert Institute for Spine Care

Sciatica pain is treatable but surgery may be required

Sciatica pain is one of the most common ailments a person can face, and if not taken seriously, it could become one of the most harmful.

...

Collins Comfort Masters

Here’s 1 way to ensure your family is drinking safe water

Water is maybe one of the most important resources in our lives, and especially if you have kids, you want them to have access to safe water.

Hackers aim to find flaws in AI – with White House help