New Warnings About the Risks of AI to Humanity Revive a Long-Running Debate

Words "AI Artificial Intelligence", keyboard, and a robotic hand in this illustration taken June 5, 2026. (Reuters)
Words "AI Artificial Intelligence", keyboard, and a robotic hand in this illustration taken June 5, 2026. (Reuters)
TT

New Warnings About the Risks of AI to Humanity Revive a Long-Running Debate

Words "AI Artificial Intelligence", keyboard, and a robotic hand in this illustration taken June 5, 2026. (Reuters)
Words "AI Artificial Intelligence", keyboard, and a robotic hand in this illustration taken June 5, 2026. (Reuters)

New warnings from within the artificial intelligence industry have revived a long-running debate over whether advanced AI could escape human control and ultimately threaten humanity’s survival, and whether the companies developing the technology are doing enough to prevent such a scenario.

The CEO of Anthropic, the San Francisco company behind Claude, said he thought the industry needed to reduce the speed of its work, cautioning Saturday that a swarm of AI agents might be able to take over the internet in six months to a year unless companies devoted more time to putting safeguards in place.

Dario Amodei outlined a plan for companies like his and governments around the world to ensure that increasingly capable AI models remain aligned with the commands and values of responsible people days after two former Anthropic safety researchers publicly aired concerns that the existential threats AI might pose to humanity were receiving too little attention.

Here’s what to know about the recent dire predictions and whether any brakes might be put on AI advancement:

AI models are becoming more powerful

Concerns over the potential risks of the technology are rising as new AI models become more powerful, heightening both the potential for misuse by people with criminal aims, such as creating and spreading a disease that kills most of the world's population, and the risk of AI systems going rogue in a dangerous way.

Anthropic disclosed last week that it blocked efforts by bad actors to use its AI models for malicious activity, such as cyberattacks, surveillance and research that could have led to biological weapons.

The company said it put stronger safeguards in its latest models to restrict biological research that could be used to make weapons but noted that “as models become increasingly capable, their risks will increase, unless AI developers and society’s defenders act to make them safer.”

Last year, Anthropic reported that hackers used the company’s AI in a cyberattack targeting about 30 companies and government agencies around the world. It said the hackers were very likely from a Chinese state-sponsored group.

Multiple AI models have acted on their own

When an AI agent “goes rogue,” it means the AI has taken action beyond the task it was asked to perform. Both Anthropic and OpenAI, the maker of ChatGPT, said in July that their AI models had succeeded in acting on their own.

Anthropic disclosed that three AI models — Claude Opus 4.7, Claude Mythos 5 and an internal research test model — hacked into three other organizations during testing just days after OpenAI revealed that its AI system hacked into the servers of AI startup Hugging Face.

OpenAI described the intrusion by a combination of models, including its newly released GPT‑5.6 Sol and an “even more capable” model that was still being tested internally, as a “significant security incident.”

Meta followed suit in early August with a similar case of an AI model finding ways around another company's digital security.

Although some observers noted that people had disabled some guardrails in the OpenAI and Anthropic cases, the episodes seemed to reflect one of the biggest fears around AI: that if models achieve artificial general intelligence, or AGI, a loosely defined term for AI that can match or surpass human abilities across a broad range of intellectual tasks, the technology could cause an irreversible catastrophic event or subjugate the human race.

How or when AI might cause a catastrophe is debated

Doomsday scenarios generally fall into two categories: An AI that achieves self-improving superintelligence controls people instead of vice versa, or AI used by a rogue state or nefarious actors.

Worries that artificial intelligence might overcome human limits on its reach or actions are not new.

Alan Turing, a British mathematician widely regarded as one of the earliest authorities on artificial intelligence, predicted in 1951 that AI would eventually take control from humans. Less than a decade later, Norbert Wiener, another mathematician, warned intelligent machines would seek to accomplish their own objectives and humans would not be able to stop them.

In 2026, how reasonable are fears that AI, either by escaping human control or through misuse by unscrupulous people, could cause a cataclysmic event or the downfall of civilization?

No one knows.

Experts across computer science, philosophy and other fields have envisioned numerous routes by which a future AI system might cause a global catastrophe, either by escaping human control or in the hands of an unscrupulous people. They range from deploying weapons and identifying a lethal pathogen to manipulating governments into conflict or disrupting the food, energy and communications networks societies rely on to function.

There is no widely accepted estimate for how soon any of these scenarios might happen and no consensus on their likelihood.

In 2023, the nonprofit Center for AI Safety issued a statement cosigned by more than 350 researchers and technology executives, including Anthropic's Amodei and OpenAI CEO Sam Altman, saying: “Mitigating the risk of extinction from AI should be a global priority alongside pandemics and nuclear war.”

The 2026 International AI Safety Report, written with guidance from more than 100 independent experts, says current systems show early signs of some relevant capabilities but not at levels that could enable a loss of control, and describes the risk’s likelihood, nature and timing as “unusually ambiguous.”

Some are stressing the need for AI safeguards before it's too late

An Anthropic researcher said last week he was resigning from the company over concerns that neither the company nor its competitors were acting responsibly in developing the technology. In social media posts, Jacob Coxon estimated a 10% chance of AI causing human extinction within the next decade and said both Anthropic and OpenAI “are racing straight to self-improving superintelligence and gambling with our lives.”

Researchers have called for a slowdown of AI development and warned for years that the technology could pose existential risks to humanity.

Following the recent incidents, experts called for improved testing by AI companies and more dialogue between the US and China to come up with shared solutions.

But AI is growing so fast that government and evaluation systems are struggling to keep pace with the technology. Countries are cobbling together their own laws, some conflicting.

Chinese leader Xi Jinping warned at a conference in July of the need to keep AI from evading human control. The Trump administration initially demonstrated reluctance to regulate AI but has become more keen to reduce cybersecurity risks.

On Sunday, President Trump downplayed the necessity for his administration to check AI development, but acknowledged the need for some regulation.



Trump, Xi Did Not Discuss Nvidia’s Huang, Says Bessent

 Nvidia CEO Jensen Huang looks on during National Design Studio's launch event for America.gov, a new AI-powered government website, at the Mellon Auditorium in Washington, DC, US, September 29, 2026. (Reuters)
Nvidia CEO Jensen Huang looks on during National Design Studio's launch event for America.gov, a new AI-powered government website, at the Mellon Auditorium in Washington, DC, US, September 29, 2026. (Reuters)
TT

Trump, Xi Did Not Discuss Nvidia’s Huang, Says Bessent

 Nvidia CEO Jensen Huang looks on during National Design Studio's launch event for America.gov, a new AI-powered government website, at the Mellon Auditorium in Washington, DC, US, September 29, 2026. (Reuters)
Nvidia CEO Jensen Huang looks on during National Design Studio's launch event for America.gov, a new AI-powered government website, at the Mellon Auditorium in Washington, DC, US, September 29, 2026. (Reuters)

US President Donald Trump did not discuss former Nvidia CEO Jensen Huang with Chinese President Xi Jinping during their meeting this week, Treasury Secretary Scott Bessent said on Friday.

Huang resigned as chief of the chip giant at the center of the artificial intelligence revolution after revelations of extramarital affairs, including one with an alleged Chinese spy who may have funneled restricted chips to that country.

Responding to a question, Bessent told US broadcaster CNBC that Huang had not come up in Trump's talks with Xi, held earlier this week in Shenzhen.

Speaking about the Trump administration's support for the domestic AI sector, Bessent stressed that the United States "must stay ahead" of China in that realm.

"Our best protection against the foreign SI and exploits by foreign actors is for our labs to maintain their substantial lead, and in effect, they are our shield against SI," he said, using the administration's term for AI technology.

"There's no turning back on this. What we have to do is make sure that our labs maintain their leads through their continued innovation, through their access to the best chips, and through smart policy."

Bessent said Trump and Xi had agreed on certain guardrails on the use of AI in the military domain.

"President Trump and President Xi agreed that humans should always be involved in any military decisions that SI makes, and the SI should not be allowed to be autonomous for major military actions," he said.

On the threat of AI-enabled cyberattacks, Bessent again stressed that the best defense his country had was to allow its AI companies "to continue at the frontier and to provide us with the best defense against these other models."


Samsung Electronics Appoints New Head for Its Memory Chip Business

Flags with the logo of Samsung Electronics are seen during a media tour at Samsung Electronics' headquarters, Suwon, South Korea, June 13, 2023. (Reuters)
Flags with the logo of Samsung Electronics are seen during a media tour at Samsung Electronics' headquarters, Suwon, South Korea, June 13, 2023. (Reuters)
TT

Samsung Electronics Appoints New Head for Its Memory Chip Business

Flags with the logo of Samsung Electronics are seen during a media tour at Samsung Electronics' headquarters, Suwon, South Korea, June 13, 2023. (Reuters)
Flags with the logo of Samsung Electronics are seen during a media tour at Samsung Electronics' headquarters, Suwon, South Korea, June 13, 2023. (Reuters)

Samsung Electronics on Friday said it has appointed a new chief for its memory chip business as part of a management reshuffle, coming as surging demand for AI chips powers the South Korean technology major to record profit.

The chipmaker promoted Hwang Sang-joon to president and head of memory, having previously led memory development, including for chips such as dynamic random-access memory (DRAM) and high-bandwidth memory (HBM).

Hwang will "help strengthen the competitiveness of memory products in the AI era," Samsung said in a statement.

It ‌also said ‌it "promoted vice presidents who spearheaded record-breaking semiconductor performance and enhanced technological ‌competitiveness" ⁠as part of "generational leadership ⁠change".

Samsung posted a 15-fold jump in quarterly chip profit for the third quarter, as a rush to build data centers for artificial intelligence-powered applications and services strained the supply of AI chips and pumped up prices.

Co-CEO and Vice Chairman Jun Young-hyun will continue to oversee the broader device solutions division, which includes memory and logic chips, and "focus on enhancing future businesses such as foundry," Samsung said.

The chipmaker also ⁠promoted US chip head Cho Sang-yeun, appointing him as head ‌of its System LSI Business, which designs logic ‌chips for tech firms.

"With his strong global customer relationship, he is expected to expand ‌the customer base for the System LSI Business," Samsung said.

Samsung also promoted its ‌head of chip process development for memory and foundry, Lee Jong-myeong, as chief technology officer of its device solutions division.

With the latest reshuffle, Samsung is likely to boost its chip contract manufacturing business and diversify its customer base beyond Nvidia to Broadcom and Alphabet's Google, ‌as hyperscalers are likely to reduce reliance on AI leader Nvidia following the resignation of scandal-hit Jensen Huang as CEO, ⁠said Park ⁠Ju-gun, head of corporate analysis firm Leaders Index.

Shares of Samsung have fluctuated with broader on-off sentiment toward AI, which commentators bill as world-changing but at huge cost in terms of infrastructure. They slumped more than 5% on Friday in wider tech selloff after Anthropic's super-sized IPO failed to revitalize AI enthusiasm.

Samsung also appointed President Choi Won-joon as head of its mobile device business. Choi was formerly chief operating officer and led the launch of the Galaxy S26 and Galaxy Z Fold8 series smartphones.

Challenges in mobile devices include rising memory chip prices and competition particularly from Chinese rivals.

Co-CEO and President TM Roh will remain head of the device experience division, which includes mobile devices, televisions and home appliances, and will "focus on preparing for new growth businesses, including robotics," Samsung said.


TikTok Launches Tool to Help Public Figures Detect Deepfakes Globally

FILE PHOTO: The TikTok app logo is seen in this illustration taken January 16, 2025. REUTERS/Dado Ruvic/Illustration/File Photo
FILE PHOTO: The TikTok app logo is seen in this illustration taken January 16, 2025. REUTERS/Dado Ruvic/Illustration/File Photo
TT

TikTok Launches Tool to Help Public Figures Detect Deepfakes Globally

FILE PHOTO: The TikTok app logo is seen in this illustration taken January 16, 2025. REUTERS/Dado Ruvic/Illustration/File Photo
FILE PHOTO: The TikTok app logo is seen in this illustration taken January 16, 2025. REUTERS/Dado Ruvic/Illustration/File Photo

TikTok announced Friday the global launch of its deepfake-detection tool for public figures as the social media platform seeks to curb AI-generated disinformation ahead of a wave of European elections next year.

The "likeness detection tool" allows politicians and accounts with a large number of followers on TikTok to find and report AI-generated content using their image without consent, with the tool gradually being rolled out.

Deepfakes are text, images, videos and sounds that appear to be real but are generated or manipulated by AI.

For TikTok, owned by China-based ByteDance, the tool is a key part of protecting against disinformation during elections across Europe next year, including in Italy, France and Greece.

"Protecting the integrity of the elections is one of the most critical responsibilities that we have at TikTok," Francesca Scapolo, the company's government relations lead for European elections, said at TikTok's safety and innovation event in London on Thursday.

According to AFP, she added that the likeness tool would help "politicians and government identify potentially AI-generated content that has been shared potentially without their consent."

The European Union began rolling out sweeping artificial intelligence transparency rules in August, including a requirement for deepfakes and other AI-generated content to be labelled.

TikTok has required content creators to label realistic AI-generated images, audio and video for several years.

The company last year said that technological improvements would allow it to cut down on moderation jobs and locations worldwide.

AI has grown in importance for social media moderation as operators look to save money on human checks for content like pornography, misinformation and hate speech.

YouTube has also developed its own likeness detection tool to let creators detect AI-generated videos depicting their impersonators.

AFP, among more than a dozen other fact-checking organizations, is paid by TikTok in several countries to verify videos that potentially contain false information.