Why captions are suddenly everywhere and how they got there

Jun 27, 2022, 12:01 AM | Updated: Jun 30, 2022, 5:52 pm

Chelle Wyatt uses her cell phone with the Otter app Friday, April 15, 2022, in Salt Lake City. Peop...

Chelle Wyatt uses her cell phone with the Otter app Friday, April 15, 2022, in Salt Lake City. People with hearing loss have adopted technology to navigate the world, especially as hearing aids are expensive and inaccessible to many. (AP Photo/Rick Bowmer)

(AP Photo/Rick Bowmer)

NEW YORK (AP) — People with hearing loss have a new ally in their efforts to navigate the world: Captions that aren’t limited to their television screens and streaming services.

The COVID pandemic disrupted daily life for people everywhere, but many of those with hearing loss took the resulting isolation especially hard. “When everyone wears a mask they are completely unintelligible to me,” said Pat Olken of Sharon, Massachusetts, whose hearing aids were insufficient. (A new cochlear implant has helped her a lot.)

So when her grandson’s bar mitzvah was streamed on Zoom early in the pandemic, well before the service offered captions, Olken turned to Otter, an app created to transcribe business meetings. Reading along with the ceremony’s speakers made the app “a tremendous resource,” she said.

People with hearing loss, a group estimated at roughly 40 million U.S. adults, have long adopted technologies to help them make their way in the hearing world, from Victorian-era ear trumpets to modern digital hearing aids and cochlear implants.

But today’s hearing aids can cost upward of $5,000, often aren’t covered by insurance and don’t work for everyone. The devices also don’t snap audible sound into focus the way glasses immediately correct vision. Instead, hearing aids and cochlear implants require the brain to interpret sound in a new way.

“The solutions out there are clearly not a one-size-fits-all model and do not meet the needs of a lot of people based on cost, access, a lot of different things,” said Frank Lin, director of the Cochlear Center for Hearing and Public Health at Johns Hopkins University. That’s not just a communication problem; researchers have found correlations between untreated hearing loss and higher risks of dementia.

Cheaper over-the-counter hearing devices are on the way. But for now, only about 20% of those who could benefit from hearing aids use one.

Captions, by contrast, are usually a lot easier to access. They’ve long been available on modern television sets and are cropping up more frequently in videoconferencing apps like Zoom, streaming services like Netflix, social media video on TikTok and YouTube, movie theaters and live arts venues.

In recent years, smartphone apps like Otter; Google’s Live Transcribe; Ava; InnoCaption, for phone calls; and GalaPro, for live theater performances, have emerged. Some are aimed at people with hearing loss and use human reviewers to make sure captions are accurate.

Others, like Otter and Live Transcribe, instead rely on what’s called automatic speech recognition, which uses artificial intelligence to learn and capture speech. ASR has issues with accuracy and lags in transcribing the spoken word; built-in biases can also make transcriptions less accurate for the voices of women, people of color and deaf people, said Christian Volger, a professor at Gallaudet University who specializes in accessible technology.

Jargon and slang can also be a stumbling block. But users and experts say that ASR has improved a lot.

While welcome, none of these solutions are perfect. Toni Iacolucci of New York says her book club could be draining even when she was using Otter to transcribe the conversation. The captions weren’t always accurate and didn’t identify individual speakers, which could make it hard to keep up, she said.

“It worked a little bit,” said Iacolucci, who lost her hearing nearly two decades ago. After coming home, she would be so tired from trying to follow the conversation that she had to lie down. “It just takes so much energy.” She got a cochlear implant a year ago that has significantly improved her ability to hear, to the point where she can now have one-on-one conversations without captions. They still help in group discussions, she said.

Otter said in a statement that it welcomes feedback from the deaf and hard of hearing community and noted that it now provides a paid software assistant that can join virtual meetings and transcribe them automatically.

Transcription lag can present other problems — among them, a worry that conversation partners might grow impatient with delays. “Sometimes you say, ‘I’m sorry, I just need to look at my captions in order to hear,'” said Richard Einhorn, a musician and composer in New York. “That doesn’t mean I’m not aware sometimes it’s a hassle for other people.”

Other issues crop up. When Chelle Wyatt of Salt Lake City went to her doctor’s office, the Wi-Fi there wasn’t strong enough for the transcription app to work. “It was gestures and writing things down and making sure I got a written report afterward so I knew what was said,” she said.

Movie theaters provide devices that amplify sound, as well as glasses and individual screens that show captions to go with the movie. But those aren’t always comfortable and sometimes aren’t well-maintained or just don’t work. Many people with hearing loss want more films to run captions on the big screen, just like you’d have in the comfort of your own home.

A new law that took effect In New York City on May 15 requires movie theaters to offer captions on the screen for up to four showtimes per movie each week, including during the most popular hours to go to the movies — Friday nights and weekends. Hawaii passed a state law in 2015 that required two screenings a week of each movie with captions on the screen. AMC, the big movie chain, also says it screens some movies with captions at about a third of its U.S. theaters.

Captions are more available now for live performances, too. Several Broadway theaters promote a smartphone app that captions live performances; there are also handheld individual devices that show captions. Theaters also have a few performances with “open captions” everyone can see.

During the pandemic, the shift to online meetings and school meant videoconferencing services became a tool of survival — but captions came only after a big push. Zoom added live transcription to its free service only in October 2021, but the meeting’s host has to enable them. Google Meet was quicker to make captions available to everyone for free in May 2020; Microsoft Teams, a workplace messaging app, did so that June.

“We need captioning everywhere and we need people to be more sensitive,” Olken said. “The more I advocate the more other people benefit.”

Copyright © The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.

AP

This photo provided by Robert Wilkes, owner of a house boat management company, shows smoke rising ...

Associated Press

Houseboats catch fire while docked at Wahweap Marina on Lake Powell

More than half a dozen house boats momentarily caught fire at a popular boating destination on the Utah-Arizona line on Friday.

1 day ago

File - Women work in a restaurant kitchen in Chicago, Thursday, March 23, 2023. On Friday, the U.S....

Associated Press

US hiring, unemployment jump in May and what that says about the economy

The nation’s employers stepped up their hiring in May, adding a robust 339,000 jobs, well above expectations.

1 day ago

(Pixabay Photo)...

Associated Press

Oath Keeper from Arizona sentenced for role in Jan. 6 riot at US Capitol

Edward Vallejo, a U.S. Army veteran from Phoenix, oversaw a “Quick Reaction Force” at a Virginia hotel that was prepared to deploy an arsenal of weapons into Washington if needed, authorities say.

2 days ago

FILE - U.S. Border Patrol Chief Raul Ortiz listens during a news conference, Jan. 5, 2023, in Washi...

Associated Press

US Border Patrol chief is retiring after seeing through end of Title 42 immigration restrictions

The head of the U.S. Border Patrol announced Tuesday that he was retiring, after seeing through a major policy shift that seeks to clamp down on illegal crossings at the U.S.-Mexico border following the end of Title 42 pandemic restrictions.

3 days ago

FILE - President Joe Biden talks with House Speaker Kevin McCarthy of Calif., on the House steps as...

Associated Press

House OKs debt ceiling bill to avoid default, sends Biden-McCarthy deal to Senate

The House approved a debt ceiling and budget cuts package late Wednesday, as President Joe Biden and Speaker Kevin McCarthy assembled a bipartisan coalition of centrist Democrats and Republicans against fierce conservative blowback and progressive dissent.

3 days ago

Sean Bickings (Family Photo via city of Tempe)...

Associated Press

Family of man who drowned last year in Tempe Town Lake files wrongful death lawsuit

The family of a man who drowned in Tempe Town Lake a year ago filed a wrongful death lawsuit against the city Wednesday, noting that its police department doesn't have a policy requiring officers to go into the water to save someone.

3 days ago

Sponsored Articles

...

DAY & NIGHT AIR CONDITIONING, HEATING AND PLUMBING

Here are the biggest tips to keep your AC bill low this summer

PHOENIX — In Arizona during the summer, having a working air conditioning unit is not just a pleasure, but a necessity. No one wants to walk from their sweltering car just to continue to be hot in their home. As the triple digits hit around the Valley and are here to stay, your AC bill […]

...

re:vitalize

Why drug-free weight loss still matters

Wanting to lose weight is a common goal for many people as they progress throughout life, but choosing between a holistic approach or to take medicine can be a tough decision.

(Photo by Michael Matthey/picture alliance via Getty Images)...

Cox Communications

Valley Boys & Girls Club uses esports to help kids make healthy choices

KTAR’s Community Spotlight focuses on the Boys & Girls Club of the Valley and the work to incorporate esports into children's lives.

Why captions are suddenly everywhere and how they got there