Spotify’s New AI-Powered Language Translation Feature Clones Your Favorite Podcaster’s Voice

Spotify has launched a pilot of Voice Translation for podcasts, using OpenAI technology to translate podcasts into other languages in the original podcaster’s voice.

While making English-language podcasts more accessible to global audiences, there are concerns about the growing use of AI translation and voice generation technologies. 

Voice Translation

Spotify announced they are piloting a new feature that translates English-spoken episodes into other languages, such as Spanish, French, and German.

The feature is powered by OpenAI’s voice generation technology to clone the original podcaster’s voice.

Spotify stated it would match the original speaker’s style, making for a more authentic listening experience that sounds more personal and natural than traditional dubbing – recording new voices for a product rather than the original voice.

Monica Padman, host of Armchair Expert, and Steven Bartlett, who hosts Diary of a CEO, are among the podcasters working with Spotify on AI-powered voice translations.

“By matching the creator’s own voice, Voice Translation gives listeners around the world the power to discover and be inspired by new podcasters in a more authentic way than ever before,” said Ziad Sultan, VP of Personalization.

However, there have been concerns over AI-powered translations and issues that have appeared using the technology.

AI Translation and Recognition Causes Concerns

Live voice translation has caused concerns over more convincing deep fakes, more effective scams, and misinterpretation. 

In 2022, refugee advocates claimed that turning to machine learning models for transition risks introducing error into asylum claims.

AI translation models are more accurate for “high-resource” languages like English than Afghan languages.

Machine translations of Dari and Pashto had led to confusion and the rejection of at least one asylum claim, Rest of the World found.

Cultural blind spots and failures to understand regional colloquialisms also introduced inaccuracies when machine translation was used to draft documents.

The Hustle also reported that the US immigration system has used AI-powered translation tools instead of human interpreters at a significant cost to asylum seekers.

Meta’s AI translation model, SeamlessM4T, found in a study that other models are two times more likely to transcribe Black speakers incorrectly.

Journalist Izadeli Montalvo warned brands against relying on AI translation, saying the connection isn’t just about language but culture and emotions: “It’s about crafting messages that resonate, not just translate.”

Featured Image Credit: Taylor Herring/PA