Lip-Syncing AI: A Potential Twist for the Business World?

It’s been a challenge for multinational firms all over the world: talking to clients and employers in their own languages. Translation apps are sometimes not ideal. And who has the time to learn a foreign language?

But some generative AI tools may be able to help—and in the process, could transform how overseas business has been conducted for decades. New tools, including an update to the dominant ChatGPT platform, can translate words into multiple languages and manipulate mouth movements to match an AI-generated script. In some cases, users and the newest version of ChatGPT can even have fluid, real-time conversations. Another tool that is gaining traction clones the voice of a user from a video they’ve uploaded and deploys lip-syncing and facial-recognition technology to create new clips of the person speaking in different languages. 

To be sure, the technology is still developing; it can’t lip-sync in real time, for example. But experts who once predicted it would be three to five years before AI-driven natural-language capabilities could catch up with actual, human-like conversation are rethinking their timetables. “Accuracy in speech recognition has been lacking, but the gaps are closing very fast,” says Chris Cantarella, global sector leader for the Software practice at Korn Ferry, noting that ChatGPT was released to the public less than a year ago. 

While voice-enabled AI is still in its infancy, the pace of its innovation and adoption means these new tools are likely to show up in the workplace sooner than expected. In a business context, generative AI that can converse in multiple languages could potentially end the spiral of misery customers encounter with chatbots today, says Paul Fogel, professional-search sector leader for software at Korn Ferry. Real-time speech translation could also help sales teams communicate with new clients and enter new markets without having to hire additional staff. Conversational AI could supplement existing mental-health and wellness offerings for staff. Marketing and communications teams could use it for podcasting, streaming, and other media that it could translate into different languages for offices around the world. 

But promising though they may be, these new tools will likely create a host of new issues for leaders to navigate. Jamen Graves, global co-leader in the CEO and Enterprise Leadership Development practice at Korn Ferry, says using lip-syncing and facial-recognition AI to translate a town-hall address could come across as an authentic attempt at inclusion—or as the exact opposite.

Problem is, these tools still have issues grasping tone, sentiment, and local idioms, which could lead to problems. “Any use of this in a business context at this point requires a person to carefully review the translation,” says Graves. “It begs the question: Are we really improving productivity and speeding things up?” 

And, of course, these new tools could—and likely will—be used to create deepfakes of corporate leaders. Ironically, though, Fogel doesn’t think the financial and reputational risk is any greater than it is now. To explain his reasoning, he points to the recent viral dental-plan ad featuring AI-generated footage of Tom Hanks that forced the actor to issue a public disavowal. “There are incredibly convincing deepfakes using AI generated voice and video already,” says Fogel. “I’m not sure this makes the risk any greater, because it’s already pretty high.”

Learn more about Korn Ferry’s Organizational Strategy capabilities.

Read more This Week in Leadership articles