Oregon dropping AI tool used in child abuse cases

Jun 2, 2022, 7:29 AM | Updated: 7:33 am

Child welfare officials in Oregon will stop using an algorithm to help decide which families are investigated by social workers, opting instead for a new process that officials say will make better, more racially equitable decisions.

The move comes weeks after an Associated Press review of a separate algorithmic tool in Pennsylvania that had originally inspired Oregon officials to develop their model, and was found to have flagged a disproportionate number of Black children for “mandatory” neglect investigations when it first was in place.

Oregon’s Department of Human Services announced to staff via email last month that after “extensive analysis” the agency’s hotline workers would stop using the algorithm at the end of June to reduce disparities concerning which families are investigated for child abuse and neglect by child protective services.

“We are committed to continuous quality improvement and equity,” Lacey Andresen, the agency’s deputy director, said in the May 19 email.

Jake Sunderland, a department spokesman, said the existing algorithm would “no longer be necessary,” since it can’t be used with the state’s new screening process. He declined to provide further details about why Oregon decided to replace the algorithm and would not elaborate on any related disparities that influenced the policy change.


This story, supported by the Pulitzer Center for Crisis Reporting, is part of an ongoing Associated Press series, “Tracked,” that investigates the power and consequences of decisions driven by algorithms on people’s everyday lives.


Hotline workers’ decisions about reports of child abuse and neglect mark a critical moment in the investigations process, when social workers first decide if families should face state intervention. The stakes are high – not attending to an allegation could end with a child’s death, but scrutinizing a family’s life could set them up for separation.

From California to Colorado and Pennsylvania, as child welfare agencies use or consider implementing algorithms, an AP review identified concerns about transparency, reliability and racial disparities in the use of the technology, including their potential to harden bias in the child welfare system.

U.S. Sen. Ron Wyden, an Oregon Democrat, said he had long been concerned about the algorithms used by his state’s child welfare system and reached out to the department again following the AP story to ask questions about racial bias – a prevailing concern with the growing use of artificial intelligence tools in child protective services.

“Making decisions about what should happen to children and families is far too important a task to give untested algorithms,” Wyden said in a statement. “I’m glad the Oregon Department of Human Services is taking the concerns I raised about racial bias seriously and is pausing the use of its screening tool.”

Sunderland said Oregon child welfare officials had long been considering changing their investigations process before making the announcement last month.

He added that the state decided recently that the algorithm would be completely replaced by its new program, called the Structured Decision Making model, which aligns with many other child welfare jurisdictions across the country.

Oregon’s Safety at Screening Tool was inspired by the influential Allegheny Family Screening Tool, which is named for the county surrounding Pittsburgh, and is aimed at predicting the risk that children face of winding up in foster care or being investigated in the future. It was first implemented in 2018. Social workers view the numerical risk scores the algorithm generates – the higher the number, the greater the risk – as they decide if a different social worker should go out to investigate the family.

But Oregon officials tweaked their original algorithm to only draw from internal child welfare data in calculating a family’s risk, and tried to deliberately address racial bias in its design with a “fairness correction.”

In response to Carnegie Mellon University researchers’ findings that Allegheny County’s algorithm initially flagged a disproportionate number of Black families for “mandatory” child neglect investigations, county officials called the research “hypothetical,” and noted that social workers can always override the tool, which was never intended to be used on its own.

Wyden is a chief sponsor of a bill that seeks to establish transparency and national oversight of software, algorithms and other automated systems.

“With the livelihoods and safety of children and families at stake, technology used by the state must be equitable — and I will continue to watchdog,” Wyden said.

The second tool that Oregon developed – an algorithm to help decide when foster care children can be reunified with their families – remains on hiatus as researchers rework the model. Sunderland said the pilot was paused months ago due to inadequate data but that there is “no expectation that it will be unpaused soon.”

In recent years while under scrutiny by a crisis oversight board ordered by the governor, the state agency – currently preparing to hire its eighth new child welfare director in six years – considered three additional algorithms, including predictive models that sought to assess a child’s risk for death and severe injury, whether children should be placed in foster care, and if so, where. Sunderland said the child welfare department never built those tools, however.


Follow Sally Ho and Garance Burke on Twitter at @_sallyho and @garanceburke.


Contact AP’s global investigative team at Investigative@ap.org or https://www.ap.org/tips/

Copyright © The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.


Haitian migrant Gerson Solay, 28, carries his daughter, Bianca, as he and his family cross into Can...
Associated Press

US, Canada to end loophole that allows asylum-seekers to move between countries

President Joe Biden and Canadian Prime Minister Justin Trudeau on Friday announced a plan to close a loophole to an immigration agreement.
1 day ago
Expert skateboarder Di'Orr Greenwood, an artist born and raised in the Navajo Nation in Arizona and...
Associated Press

Indigenous skateboard art featured on new stamps unveiled at Phoenix skate park

The Postal Service unveiled the “Art of the Skateboard" stamps at a Phoenix skate park, featuring designs from Indigenous artists.
1 day ago
(Facebook Photo/City of San Luis, Arizona)...
Associated Press

San Luis authorities receive complaints about 911 calls going across border

Authorities in San Luis say they are receiving more complaints about 911 calls mistakenly going across the border.
7 days ago
(Pexels Photo)...
Associated Press

Daylight saving time begins in most of US this weekend

No time change is observed in Hawaii, most of Arizona, Puerto Rico, the U.S. Virgin Islands, American Samoa, Guam and the Northern Marianas.
15 days ago
Mexican army soldiers prepare a search mission for four U.S. citizens kidnapped by gunmen in Matamo...
Associated Press

How the 4 abducted Americans in Mexico were located

The anonymous tip that led Mexican authorities to a remote shack where four abducted Americans were held described armed men and blindfolds.
15 days ago
Tom Brundy points to a newly built irrigation canal on one of the fields at his farm Tuesday, Feb. ...
Associated Press

Southwest farmers reluctant to idle farmland to save water

There is a growing sense that fallowing will have to be part of the solution to the increasingly desperate drought in the West.
22 days ago

Sponsored Articles

(Photo: OCD & Anxiety Treatment Center)...

Here’s what you need to know about OCD and where to find help

It's fair to say that most people know what obsessive-compulsive spectrum disorders generally are, but there's a lot more information than meets the eye about a mental health diagnosis that affects about one in every 100 adults in the United States.
(Photo by Michael Matthey/picture alliance via Getty Images)...
Cox Communications

Valley Boys & Girls Club uses esports to help kids make healthy choices

KTAR’s Community Spotlight focuses on the Boys & Girls Club of the Valley and the work to incorporate esports into children's lives.
Fiesta Bowl Foundation

Celebrate 50 years of Vrbo Fiesta Bowl Parade magic!

Since its first production in the early 1970s, the Vrbo Fiesta Bowl Parade presented by Lerner & Rowe has been a staple of Valley traditions, bringing family fun and excitement to downtown Phoenix.
Oregon dropping AI tool used in child abuse cases