Can AI Make Video Games More Immersive? Some Studios Turn to AI-Fueled NPCs for More Interaction

The AI (Artificial Intelligence) letters and robot hand are placed on computer motherboard in this illustration taken, June 23, 2023. (Reuters)
The AI (Artificial Intelligence) letters and robot hand are placed on computer motherboard in this illustration taken, June 23, 2023. (Reuters)
TT

Can AI Make Video Games More Immersive? Some Studios Turn to AI-Fueled NPCs for More Interaction

The AI (Artificial Intelligence) letters and robot hand are placed on computer motherboard in this illustration taken, June 23, 2023. (Reuters)
The AI (Artificial Intelligence) letters and robot hand are placed on computer motherboard in this illustration taken, June 23, 2023. (Reuters)

For decades, video games have relied on scripted, stilted interactions with non-player characters to help shepherd gamers in their journeys. But as artificial intelligence technology improves, game studios are experimenting with generative AI to help build environments, assist game writers in crafting NPC dialogue and lend video games the improvisational spontaneity once reserved for table-top role-playing games.

In the multiplayer game “Retail Mage,” players help run a magical furniture store and assist customers in hopes of earning a five-star review. As a salesperson — and wizard — they can pick up and examine items or tell the system what they'd like to do with a product, such as deconstruct chairs for parts or tear a page from a book to write a note to a shopper.

A player’s interactions with the shop and NPCs around them — from gameplay mechanics to content and dialogue creation — are fueled by AI rather than a predetermined script to create more options for chatting and using objects in the shop.

“We believe generative AI can unlock a new kind of gameplay where the world is more responsive and more able to meet players at their creativity and the things that they come up with and the stories they want to tell inside a fantasy setting that we create for them,” said Michael Yichao, cofounder of Jam & Tea Studios, which created “Retail Mage.”

The typical NPC experience often leaves something to be desired. Pre-scripted interactions with someone meant to pass along a quest typically come with a handful of chatting options that lead to the same conclusion: players get the information they need and continue on. Game developers and AI companies say that by using generative AI tech, they aim to create a richer experience that allows for more nuanced relationships with the people and worlds that designers build.

Generative AI could also provide more opportunities for players to go off-script and create their own stories if designers can craft environments that feel more alive and can react to players' choices in real-time.

Tech companies continue to develop AI for games, even as developers debate how, and whether, they’ll use AI in their products. Nvidia created its ACE technologies to bring so-called “digital humans” to life with generative AI. Inworld AI provides developers with a platform for generative NPC behavior and dialogue. Gaming company Ubisoft said last year that it uses Ghostwriter, an in-house AI tool, to help write some NPC dialogue without replacing the video game writer.

A report released by the Game Developers Conference in January found that nearly half of developers surveyed said generative AI tools are currently being used in their workplace, with 31% saying they personally use those tools. Developers at indie studios were most likely to use generative AI, with 37% reporting using the tech.

Still, roughly four out of five developers said they worry about the ethical use of AI. Carl Kwoh, Jam & Tea's CEO, said AI should be used responsibly alongside creators to elevate stories — not to replace them.

“That’s always been the goal: How can we use this tool to create an experience that makes players more connected to each other?” said Kwoh, who is also one of the company’s founders. “They can tell stories that they couldn’t tell before.”

Using AI to provide NPCs with endless things to say is “definitely a perk,” Yichao said, but "content without meaning is just endless noise." That's why Jam & Tea uses AI — through Google's Gemma 2 and their own servers in Amazon — to give NPCs the ability to do more than respond, he said. They can look for objects as they’re shopping or respond to other NPCs to add “more life and reactivity than a typically scripted encounter.”

“I’ve watched players turn our shopping experience into a bit of a dating sim as they flirt with customers and then NPCs come up with very realistic responses,” he said. “It’s been really fun to see the game react dynamically to what players bring to the table.”

Demonstrating a conversation with an NPC in the game “Mecha BREAK,” in which players battle war machines, Ike Nnole said that Nvidia has made its AI “humans” respond faster than they previously could by using small language models. Using Nvidia's AI, players can interact with the mechanic, Martel, by asking her to do things like customize the color of a mech machine.

“Typically, a gamer would go through menus to do all this,” Nnole, a senior product marketing manager at Nvidia said. “Now it could be a much more interactive, much quicker experience.”

Artificial Agency, a Canadian AI company, built an engine that allows developers to bring AI into any part of their game — not only NPCs, but also companions and “overseer agents” that can steer a player towards content they’re missing. The AI can also create tutorials to teach players a skill that they are missing so they can have more fun in-game, the company said.

“One way we like to put it is putting a game designer on the shoulder of everyone as they’re playing the game,” said Alex Kearney, cofounder of Artificial Agency. The company’s AI engine can be integrated at any stage of the game development cycle, she said.

Brian Tanner, Artificial Agency's CEO, said scripting every possible outcome of a game can be tedious and difficult to test. Their system allows designers to act more like directors, he said, by telling characters more about their motivation and background.

"These characters can improvise on the spot depending on what’s actually happening in the game,” Tanner said.

It's easy to run into a game's guardrails, Tanner said, where NPCs keep repeating the same phrase regardless of how players interact with them. But as AI continues to evolve, that will change, he added.

“It is truly going to feel like the world’s alive and like everything really reacts to exactly what’s happening," he said. “That’s going to add tremendous realism.”



Inside the Underground Lab in China Tasked with Solving a Physics Mystery

A view of the soon-to-be-completed and sealed central detector at the Jiangmen Underground Neutrino Observatory (JUNO), during an organized media tour by the Chinese foreign ministry and the Chinese Academy of Sciences (CAS), in Kaiping, Guangdong province, China October 11, 2024. (Reuters)
A view of the soon-to-be-completed and sealed central detector at the Jiangmen Underground Neutrino Observatory (JUNO), during an organized media tour by the Chinese foreign ministry and the Chinese Academy of Sciences (CAS), in Kaiping, Guangdong province, China October 11, 2024. (Reuters)
TT

Inside the Underground Lab in China Tasked with Solving a Physics Mystery

A view of the soon-to-be-completed and sealed central detector at the Jiangmen Underground Neutrino Observatory (JUNO), during an organized media tour by the Chinese foreign ministry and the Chinese Academy of Sciences (CAS), in Kaiping, Guangdong province, China October 11, 2024. (Reuters)
A view of the soon-to-be-completed and sealed central detector at the Jiangmen Underground Neutrino Observatory (JUNO), during an organized media tour by the Chinese foreign ministry and the Chinese Academy of Sciences (CAS), in Kaiping, Guangdong province, China October 11, 2024. (Reuters)

A giant sphere 700 m (2,300 ft) underground with thousands of light-detecting tubes will be sealed in a 12-storey cylindrical pool of water in coming months for an experiment that will shine new light on elusive subatomic particles known as neutrinos.

After years of construction, the $300 million Jiangmen Underground Neutrino Observatory (JUNO) in China's southern Guangdong province will soon start gathering data on neutrinos, a product of nuclear reactions, to help solve one of the biggest mysteries in particle physics.

Every second, trillions of extremely small neutrinos pass through matter, including the human body. In mid-flight, a neutrino, of which there are three known varieties, could transform into other types. Determining which types are the lightest and the heaviest would offer clues to subatomic processes during the early days of the universe and to explaining why matter is the way it is.

To that end, Chinese physicists and collaborating scientists from all over the world will analyze the data on neutrinos emitted by two nearby Guangdong nuclear power plants for up to six years.

JUNO would also be able to observe neutrinos from the sun, gaining a real-time view of solar processes. It could also study neutrinos given off by the radioactive decay of uranium and thorium in the Earth to better understand mantle convection driving tectonic plates.

Due to go operational in the latter half of 2025, JUNO will outpace the far larger Deep Underground Neutrino Experiment (DUNE) under construction in the United States. DUNE, backed by the Long-Baseline Neutrino Facility (LBNF) under the US Department of Energy's (DOE) top particle physics laboratory, Fermilab, will come online around 2030.

The race to understand neutrinos and advance the study of particle physics, which has transformed medical imaging technologies and developed new energy sources, intensified when the DOE abruptly cut funding for US institutes collaborating on JUNO. It instead focused on building DUNE, which has since been plagued by delays and budget overruns, with costs skyrocketing to more than $3 billion.

"China had supported Fermilab's LBNF at the time, but later the cooperation could not continue," Wang Yifang, chief scientist and project manager of JUNO, told Reuters during a recent government-backed media tour of the facility.

"Around 2018-2019, the US DOE asked all national laboratories not to cooperate with China, so Fermilab was forced to stop working with us."

The DOE, the largest US funding agency for particle physics, did not respond to Reuters' request for comment.

Sino-US tensions have risen sharply over the past decade. A trade war erupted during the Trump administration and President Joe Biden later cracked down on the sale of advanced technology to China.

In August, a bilateral science and technology cooperation pact signed in 1979 lapsed, potentially pushing more scientists to seek alternative partners, creating duplication in research and missing out on collaboration that otherwise might have led to beneficial discoveries.

In the 2010s, the countries jointly produced a nuclear reactor that could use low-enriched uranium, minimizing the risk of any fuel being weaponized.

China's foreign ministry said Beijing was "in communication" with Washington about the lapsed science agreement. The US State Department did not comment.

SOLE US COLLABORATOR

Institutions collaborating on JUNO hail from locations including France, Germany, Italy, Russia and the US, and even self-governed Taiwan, which China claims as part of its territory.

Neutrino observatories are also being constructed in other places.

"The one in the US will be six years behind us. And the one in the France and in Japan, they will be two or three years later than us. So we believe that we can get the result of mass hierarchy (of neutrinos) ahead of everybody," Wang said.

So far, real-life neutrino applications remain a distant prospect. Some scientists have mulled the possibility of relaying long-distance messages via neutrinos, which pass through solid matter such as the Earth at near light speed.

Researchers are keeping their distance from politics to focus on science, although they remain at the mercy of governments providing the funding.

One US group remains in JUNO, backed by the National Science Foundation, which recently renewed its funding for its collaboration for another three years, the group's leading physicist told Reuters.

In contrast, more than a dozen US institutes participated in the predecessor to JUNO, the Daya Bay experiment, also in Guangdong.

"Despite any political differences, I believe that through our collaboration on this scientific endeavor, we are setting a positive example that may contribute, even in a small way, to bringing our countries closer together," said J. Pedro Ochoa-Ricoux of the University of California, Irvine.

DATA INTEGRITY

The passage of neutrinos from the two power stations will be logged by JUNO's 600 metric ton spherical detector, which will immediately transmit the data to Beijing electronically. The data will be simultaneously relayed to Russia, France and Italy, where it can be accessed by all of the collaborating institutions, said Cao Jun, JUNO's deputy manager.

Data integrity has been a concern among foreign companies in China since a law was enacted in 2021 on the use, storage and transfer of data in the name of safeguarding national security.

"We have a protocol to make sure that no data is missing," Cao said.

For data on the more crucial aspects of the experiment, at least two independent teams will conduct analyses, with their results cross-checked.

"When these two groups get a consistent result, we can publish it," Cao said.

US-based Ochoa-Ricoux, who previously collaborated on China's Daya Bay experiment, will lead the data analysis for JUNO. He will also be involved in the DUNE data analysis.

"We welcome the Americans," said Wang, also director of the Institute of High Energy Physics, the Chinese counterpart of Fermilab.