Oregon dropping AI tool used in child abuse cases

Jun 2, 2022, 7:29 AM | Updated: 7:33 am

Child welfare officials in Oregon will stop using an algorithm to help decide which families are investigated by social workers, opting instead for a new process that officials say will make better, more racially equitable decisions.

The move comes weeks after an Associated Press review of a separate algorithmic tool in Pennsylvania that had originally inspired Oregon officials to develop their model, and was found to have flagged a disproportionate number of Black children for “mandatory” neglect investigations when it first was in place.

Oregon’s Department of Human Services announced to staff via email last month that after “extensive analysis” the agency’s hotline workers would stop using the algorithm at the end of June to reduce disparities concerning which families are investigated for child abuse and neglect by child protective services.

“We are committed to continuous quality improvement and equity,” Lacey Andresen, the agency’s deputy director, said in the May 19 email.

Jake Sunderland, a department spokesman, said the existing algorithm would “no longer be necessary,” since it can’t be used with the state’s new screening process. He declined to provide further details about why Oregon decided to replace the algorithm and would not elaborate on any related disparities that influenced the policy change.


This story, supported by the Pulitzer Center for Crisis Reporting, is part of an ongoing Associated Press series, “Tracked,” that investigates the power and consequences of decisions driven by algorithms on people’s everyday lives.


Hotline workers’ decisions about reports of child abuse and neglect mark a critical moment in the investigations process, when social workers first decide if families should face state intervention. The stakes are high – not attending to an allegation could end with a child’s death, but scrutinizing a family’s life could set them up for separation.

From California to Colorado and Pennsylvania, as child welfare agencies use or consider implementing algorithms, an AP review identified concerns about transparency, reliability and racial disparities in the use of the technology, including their potential to harden bias in the child welfare system.

U.S. Sen. Ron Wyden, an Oregon Democrat, said he had long been concerned about the algorithms used by his state’s child welfare system and reached out to the department again following the AP story to ask questions about racial bias – a prevailing concern with the growing use of artificial intelligence tools in child protective services.

“Making decisions about what should happen to children and families is far too important a task to give untested algorithms,” Wyden said in a statement. “I’m glad the Oregon Department of Human Services is taking the concerns I raised about racial bias seriously and is pausing the use of its screening tool.”

Sunderland said Oregon child welfare officials had long been considering changing their investigations process before making the announcement last month.

He added that the state decided recently that the algorithm would be completely replaced by its new program, called the Structured Decision Making model, which aligns with many other child welfare jurisdictions across the country.

Oregon’s Safety at Screening Tool was inspired by the influential Allegheny Family Screening Tool, which is named for the county surrounding Pittsburgh, and is aimed at predicting the risk that children face of winding up in foster care or being investigated in the future. It was first implemented in 2018. Social workers view the numerical risk scores the algorithm generates – the higher the number, the greater the risk – as they decide if a different social worker should go out to investigate the family.

But Oregon officials tweaked their original algorithm to only draw from internal child welfare data in calculating a family’s risk, and tried to deliberately address racial bias in its design with a “fairness correction.”

In response to Carnegie Mellon University researchers’ findings that Allegheny County’s algorithm initially flagged a disproportionate number of Black families for “mandatory” child neglect investigations, county officials called the research “hypothetical,” and noted that social workers can always override the tool, which was never intended to be used on its own.

Wyden is a chief sponsor of a bill that seeks to establish transparency and national oversight of software, algorithms and other automated systems.

“With the livelihoods and safety of children and families at stake, technology used by the state must be equitable — and I will continue to watchdog,” Wyden said.

The second tool that Oregon developed – an algorithm to help decide when foster care children can be reunified with their families – remains on hiatus as researchers rework the model. Sunderland said the pilot was paused months ago due to inadequate data but that there is “no expectation that it will be unpaused soon.”

In recent years while under scrutiny by a crisis oversight board ordered by the governor, the state agency – currently preparing to hire its eighth new child welfare director in six years – considered three additional algorithms, including predictive models that sought to assess a child’s risk for death and severe injury, whether children should be placed in foster care, and if so, where. Sunderland said the child welfare department never built those tools, however.


Follow Sally Ho and Garance Burke on Twitter at @_sallyho and @garanceburke.


Contact AP’s global investigative team at Investigative@ap.org or https://www.ap.org/tips/

Copyright © The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.


FILE - Police officers stand outside a Target store as a group of people protest across the street,...

Associated Press

Pride becomes a minefield for big companies, but many continue their support

Many big companies, including Target and Bud Light's parent, are still backing Pride events in June despite the minefield that the monthlong celebration has become for some of them.

2 days ago

FILE - Then-Democratic presidential candidate Joe Biden plays music on a phone as he arrives to spe...

Associated Press

Biden, looking to shore up Hispanic support, faces pressure to get 2024 outreach details right

Joe Biden vowed in 2020 to work “like the devil” to energize Hispanic voters, and flew to Florida seven weeks before Election Day to do just that.

2 days ago

Editorial members of the Austin American-Statesman's Austin NewsGuild picket along the Congress Ave...

Associated Press

Correction: US-Gannett Walkout story

Journalists at two dozen local newspapers across the U.S. walked off the job Monday to demand an end to painful cost-cutting measures and a change of leadership at Gannett, the country's biggest newspaper chain.

2 days ago

FILE - The logo of the Organization of the Petroleoum Exporting Countries (OPEC) is seen outside of...

Associated Press

Saudi Arabia reducing global oil supply, could spell higher prices for US drivers

Saudi Arabia will reduce how much oil it sends to the global economy, taking a unilateral step to prop up the sagging price of crude.

3 days ago

This photo provided by Robert Wilkes, owner of a house boat management company, shows smoke rising ...

Associated Press

Houseboats catch fire while docked at Wahweap Marina on Lake Powell

More than half a dozen house boats momentarily caught fire at a popular boating destination on the Utah-Arizona line on Friday.

5 days ago

File - Women work in a restaurant kitchen in Chicago, Thursday, March 23, 2023. On Friday, the U.S....

Associated Press

US hiring, unemployment jump in May and what that says about the economy

The nation’s employers stepped up their hiring in May, adding a robust 339,000 jobs, well above expectations.

5 days ago

Sponsored Articles



Thank you to Al McCoy for 51 years as voice of the Phoenix Suns

Sanderson Ford wants to share its thanks to Al McCoy for the impact he made in the Valley for more than a half-decade.



Why drug-free weight loss still matters

Wanting to lose weight is a common goal for many people as they progress throughout life, but choosing between a holistic approach or to take medicine can be a tough decision.

(Photo: OCD & Anxiety Treatment Center)...

OCD & Anxiety Treatment Center

Here’s what you need to know about OCD and where to find help

It's fair to say that most people know what obsessive-compulsive spectrum disorders generally are, but there's a lot more information than meets the eye about a mental health diagnosis that affects about one in every 100 adults in the United States.

Oregon dropping AI tool used in child abuse cases