عربي English Türkçe اردو فارسى
Logo
Latest News
Technology

Is that Tom Hanks Speaking in Japanese? No, it’s Just AI

Tom Hanks. (AP)
Tom Hanks. (AP)
TT
20
TT

Is that Tom Hanks Speaking in Japanese? No, it’s Just AI

Tom Hanks. (AP)
Tom Hanks. (AP)

Bad lip-syncing in dubbing and subtitles can put off audiences and hurt box office takings of foreign films.

AI may be about to change all that.

Start-up Flawless AI, co-founded by film director Scott Mann, has a tool that it says can accurately recreate lip sync in dubbing without altering the performance of the actors.

The tool studies how actors move their mouths and swaps the movements out according to the dubbed words in different languages, making it seem like Tom Hanks can speak Japanese or Jack Nicholson is fluent in French.

Mann was inspired to come up with the tool when he saw how dubbing affected the narrative cohesion of his 2015 film "Heist", starring Robert De Niro.

"I hate dubbing as it stands," he told Reuters. "You have to change so many things to try and catch sync. You're changing words filmmakers and performers have thought about so deeply. They're thrown out to find a different word that fits, but it never really does."

Mann decided to do something about this conspicuous mismatch. After some research, he discovered a white paper by Christian Theobalt of the Max Planck Institute for Informatics, laying out a new approach that uses artificial intelligence to recreate photo-real human visual effects.

The tool worked to translate a De Niro scene in "Heist".

"It's able to essentially take an 'ooh' sound from De Niro 20 minutes earlier and place that in a different moment in the film," explained Mann. "It measures at the same time and blends it so that the performance is the same but it's a different mouth movement."

Flawless AI is now working with producers and studios to integrate the technology into post-production. The first films using this tool are around a year away.

Mann believes this technology will be a game-changer, helping to eradicate foreign-language categories at award shows and creating a more inclusive film industry with a real global stage.

"From a filmmaking point of view, you're going to see the rise of a much more diverse range of stars," he said.

There might be a dissonance and a focus on the AI technology in the beginning when it is introduced, but eventually Mann hopes its use will be seamless.

"When it's a common thing, we're not going to think about it anymore and we'll just enjoy the movies," he said.

Most Viewed

 
Technology

Taiwan Cultivates Young Overseas Chip Talent with Summer Camps, University Courses

Students dressed in white protective suit and a face mask visit a clean room as part of a summer camp organized by US chip designer Synopsys with the goal to attract more youth to Taiwan's semiconductor industry, in Hsinchu, Taiwan July 18, 2025. (Reuters)
Students dressed in white protective suit and a face mask visit a clean room as part of a summer camp organized by US chip designer Synopsys with the goal to attract more youth to Taiwan's semiconductor industry, in Hsinchu, Taiwan July 18, 2025. (Reuters)
TT
20
TT

Taiwan Cultivates Young Overseas Chip Talent with Summer Camps, University Courses

Students dressed in white protective suit and a face mask visit a clean room as part of a summer camp organized by US chip designer Synopsys with the goal to attract more youth to Taiwan's semiconductor industry, in Hsinchu, Taiwan July 18, 2025. (Reuters)
Students dressed in white protective suit and a face mask visit a clean room as part of a summer camp organized by US chip designer Synopsys with the goal to attract more youth to Taiwan's semiconductor industry, in Hsinchu, Taiwan July 18, 2025. (Reuters)

Dressed in a white protective suit and face mask, Nicolas Chueh listened intently as a guide introduced a series of silver machines used in manufacturing Taiwan's cutting-edge semiconductors. The 16-year-old was among students from eight countries at the summer camp staged to raise interest in Taiwan's most vital industry amid a fast-declining birth rate that could leave tens of thousands of critical jobs vacant.

"I myself really enjoy playing video games. So I'm really just always using these semiconductor products," said Chueh, whose parents enrolled him after he expressed interest.

The camp, organized by US chip design software firm Synopsys, is among several such events staged by chip companies and Taiwanese universities in recent years as demand for semiconductors, which power most electronics and AI servers, surges across the globe.

But for the first time this year, Synopsys, which has significant operations in Taiwan to be closer to the semiconductor supply chain, hosted the events both in Mandarin and English as Taiwan searches for overseas talent.

"There is an urgent need to strengthen STEM education from an early age," said Robert Li, Synopsys's Taiwan chairman, who believes the camps can increase interest in the chip industry and help prime some of its future leaders.

"That is why we are launching this initiative in Taiwan, where its strength in semiconductors meets the challenge of demographic decline. Taken together, it is clear we must act here first."

Given limitations posed by Taiwan's ageing population, Synopsys is also considering hosting camps internationally to spur interest in chip making and designing, he added. The company charges T$33,000 ($1,103) for the English versions and T$10,900 for Mandarin.

Chueh, a dual Taiwan-Belgian national who lives in Singapore, said he views semiconductors as an attractive career choice.

"I want to lean into it to some extent because I think it will be crucial in the future with AI."

SLUMPING BIRTH RATE

Taiwan, which has a population of around 23 million, holds outsized influence over the global semiconductor supply chain, thanks to its chip companies such as Taiwan Semiconductor Manufacturing Co, the world's largest contract chipmaker, MediaTek, and UMC.

Any decline in the industry poses an existential threat to Taiwan, which faces the threat of invasion from Beijing and draws much of its global significance from the chip behemoths.

But job openings in the semiconductor sector have risen from 19,401 in the second quarter of 2020 to 33,725 in the same period this year, according to 104 Corporation, a local human resources firm.

The industry is grappling with a shortage of both highly skilled professionals, such as IC design and semiconductor R&D engineers, and essential production staff, including operators and assembly technicians.

Filling those jobs locally is becoming harder each year as Taiwan's annual number of births has dropped from over 210,000 in 2014 to around 135,000 in 2024, according to government statistics. STEM graduates have also fallen by around 15% in that period, Ministry of Education statistics showed.

"Growth in Taiwan's semiconductor industry has been quite rapid, faster than what our schools can produce in terms of engineering talent each year," said Leuh Fang, chairman of Vanguard International Semiconductor, a Taiwan-based chipmaker affiliated with TSMC.

'THE FUTURE WORKFORCE'

Last year, the National Taiwan University launched a global undergraduate semiconductor program for foreign students, which included Mandarin courses to help them reach the proficiency needed to stay and work in Taiwan.

The program now enrolls over 40 students from more than 10 countries.

TSMC also began looking toward foreign talent by throwing its weight behind a program in Germany’s Saxony state, which would send German students to study for a semester at Taiwanese universities before interning at TSMC.

Other initiatives are attempting to create interest among children as young as 10.

Taiwan's National Yang Ming Chiao Tung University (NYCU) launched an outreach program in July, backed by TSMC, aimed at making chip science fun through interactive teaching tools and online games.

"The issue everyone is discussing now is where the future workforce will come from," said NYCU President Chi-Hung Lin.

"If they're curious now, they won't reject it later and some may even grow to like this kind of work."