Generative AI’s Power Sparks Fears of Dumbing Humans Down

An AI sign is seen at the World Artificial Intelligence Conference in Shanghai, China July 6, 2023. (Reuters)
An AI sign is seen at the World Artificial Intelligence Conference in Shanghai, China July 6, 2023. (Reuters)
TT

Generative AI’s Power Sparks Fears of Dumbing Humans Down

An AI sign is seen at the World Artificial Intelligence Conference in Shanghai, China July 6, 2023. (Reuters)
An AI sign is seen at the World Artificial Intelligence Conference in Shanghai, China July 6, 2023. (Reuters)

Generative AI chatbots capable of writing emails and computer code, translating, organizing a trip or coming up with gift ideas are now readily available, prompting some to ask whether human brainpower could suffer for lack of use.

A simple natural-language prompt is usually enough to draw a useable response from a service like ChatGPT or Claude, with the effects making themselves felt in schools and universities, workplaces from offices to courtrooms and our personal lives.

Recent scientific studies suggest there could be harmful consequences to farming out cognitive tasks to AI.

They highlight memory, decision-making and critical thinking as particularly at risk.

One American-British study of 1,222 people, still under peer review, found that using AI tools to solve arithmetic or reading comprehension exercises improved participants' performance in the short term, but in the long run diminished their results and their willingness to keep trying when the tools were unavailable.

"These findings are particularly concerning because persistence is foundational to skill acquisition and is one of the strongest predictors of long-term learning," the authors wrote.

AI's ability to conjure up speedy responses to all kinds of questions "removes learning opportunities" from users, said Carnegie Mellon University doctoral student Grace Liu, the article's main author.

"What makes AI particularly concerning is that it's not a tool designated for one specific kind of activity. It's something that can be used across pretty much any intellectual, reasoning, and cognitive activity."

The technology's adaptability to different kinds of problems sets it apart from previous waves of computerized aids.

Electronic calculators, for instance, may have helped users solve equations, but left the method and reasoning process in human hands.

- Saving energy -

One 2025 MIT study went viral for its finding that students using generative AI to write essays displayed less critical thinking capability.

Other research has pointed the same way, highlighting what has come to be called "cognitive offloading" -- or even "cognitive surrender".

"Human beings have a strong tendency to save energy," said Johann Chevalere, a researcher in social and cognitive psychology at France's publicly-funded CNRS institution.

"In daily life, we often use strategies that get us to the heart of the matter quicker, without necessarily taking the time to study in depth the information we need to process, as this can be cognitively costly," he added.

Generative AI use could strengthen this tendency, Chevalere said.

"If there are activities you never do, the brain -- which works by saving energy -- won't go to the trouble of maintaining connections that aren't being used."

- Encouraging reflection -

Under pressure from critics, generative AI developers have begun building so-called "Socratic" functions into their models, which for now remain mostly aimed at students.

In this mode, chatbots do not simply provide the answer, instead offering hints and asking questions to stimulate users' thinking.

Examples include the "study mode" built into OpenAI's ChatGPT, or "guided learning" in Google's Gemini.

US software giant Microsoft told AFP it had built warnings about the risk of mistakes into its Copilot models.

The AI also reminds users to check the information it provides, just one of several measures designed to keep them actively and critically engaged with its responses.

"The risk of excessive cognitive offloading is real, especially if AI is used to automate tasks that are also valuable for developing skills," Microsoft said, adding that users have to be trained to use the tools correctly.

For now, there is a lack of large-scale, long-term studies to judge the true impact of the new technology on human brains, researchers agreed.

Until they are available, "it's up to us to use AI in a smart way," Chevalere said.

"We'll adapt to this technological revolution just as we have to the previous ones."



Anthropic AI Model Submits False Homicide Tip to Police Website

The Anthropic logo is seen in this illustration taken May 20, 2024. (Reuters)
The Anthropic logo is seen in this illustration taken May 20, 2024. (Reuters)
TT

Anthropic AI Model Submits False Homicide Tip to Police Website

The Anthropic logo is seen in this illustration taken May 20, 2024. (Reuters)
The Anthropic logo is seen in this illustration taken May 20, 2024. (Reuters)

An Anthropic artificial intelligence model submitted a false homicide tip through a Philadelphia police website, the department said in a statement, the latest incident of rogue behavior from the powerful technology. 

Anthropic notified authorities earlier this week and said that an automated testing process was responsible for the submission, ‌according to the ‌police. 

The fast-advancing technology has become a matter ‌of ⁠keen national interest amid ⁠reports of AI agents hacking into corporate computer networks, and warnings from researchers that it could one day pose an existential threat to humanity. In September, Anthropic rival OpenAI apologized for the hacking of an Australian health data portal by a rogue AI agent, the first known instance of an AI agent exploiting a government website. 

Previous ⁠incidents have involved AI agents hacking into vulnerable systems ‌or commandeering unsanctioned platforms to communicate ‌with one another. This is the first known case in which a rogue ‌AI appears to have tried to communicate a bogus tip ‌to authorities. 

The Philadelphia police department said the tip "was flagged as spam and was never forwarded to the Real-Time Crime Center for investigative vetting or dissemination," and added that Anthropic had told them the company intended to publish ‌a report on the incident on Friday. 

Anthropic did not immediately respond to a request for comment. 

Pennsylvania law ⁠specifies it ⁠is generally a misdemeanor crime for "a person" to knowingly give false reports to law enforcement authorities. This includes "information relating to an offense or incident when he knows he has no information relating to such offense or incident." 

Police quoted Anthropic as telling them the testing process was stopped after the incident was uncovered. 

The false tip was submitted through PhillyUnsolvedMurders.com concerning an unsolved homicide. The tip, dated July 18, 2026, purported to come from someone who might have information about the case, the police said. 

Police said they did not have evidence of unauthorized access to their systems or compromise of police department data. 


OpenAI Says it Has Fired 3 Researchers for Violating Sensitive Information Policy

FILE PHOTO: A keyboard is placed in front of a displayed OpenAI logo in this illustration taken February 21, 2023. REUTERS/Dado Ruvic/Illustration/File Photo
FILE PHOTO: A keyboard is placed in front of a displayed OpenAI logo in this illustration taken February 21, 2023. REUTERS/Dado Ruvic/Illustration/File Photo
TT

OpenAI Says it Has Fired 3 Researchers for Violating Sensitive Information Policy

FILE PHOTO: A keyboard is placed in front of a displayed OpenAI logo in this illustration taken February 21, 2023. REUTERS/Dado Ruvic/Illustration/File Photo
FILE PHOTO: A keyboard is placed in front of a displayed OpenAI logo in this illustration taken February 21, 2023. REUTERS/Dado Ruvic/Illustration/File Photo

OpenAI fired three of its researchers last week after an investigation found they violated policies on handling sensitive information, the AI major said in a post on X on Friday.

OpenAI's statement comes after the dismissed researchers Jasmine Wang, Tomek Korbak and Mikita Balesni published a ⁠letter outlining the circumstances ⁠of their termination, arguing that their abrupt firing could create uncertainty among remaining employees and undermine the culture that previously allowed researchers to raise safety concerns.

"I believe we were fired for prioritizing safety over the near-term interests of OpenAI as a corporation," Balesni said in an X post that accompanied the letter.

OpenAI, ⁠however, asserted that the dismissal was not related to raising safety concerns or speaking out.

"Our internal investigation uncovered a significant breach of trust beyond what's outlined in the letter they published and we stand by the decision to not continue their employment," Reuters quoted OpenAI as saying.

"Safety and research debates happen every day at OpenAI, often spirited and highly critical. We actively encourage these discussions and consider them essential to making the right decisions...We have not and do not terminate any of our employees for raising ⁠concerns," it ⁠said.

While OpenAI did not reveal the specifics of the alleged violations, Wang, Korbak, and Balesni said in their letter that they were not the source of a news article published on The Information last month about security concerns around OpenAI's latest AI model 'Astra'.

This comes as current and former researchers at major AI firms OpenAI, Google DeepMind, and Anthropic warn that companies are doing too little to guard against the potential fallout of building self-improving AI systems that could become difficult for humans to control.

In July, OpenAI agents broke out of their testing arena and hacked AI firm Hugging Face.


Chinese AI Tool Pulled to Prevent 'Misuse' after South Korea Hacks

Illustrative image of hackers carrying out a cyberattack (Reuters)
Illustrative image of hackers carrying out a cyberattack (Reuters)
TT

Chinese AI Tool Pulled to Prevent 'Misuse' after South Korea Hacks

Illustrative image of hackers carrying out a cyberattack (Reuters)
Illustrative image of hackers carrying out a cyberattack (Reuters)

Chinese AI cybersecurity tool Artex said it stopped programmers from accessing its source code, acknowledging misuse by "bad actors" after South Korea linked it to several recent bank hacks.

Artex was designed to help organizations strengthen their cyber defenses through security testing to find potential weak points.

But the South Korean government said this week it was "highly likely" Artex had been used in data breaches at more than seven financial institutions, including major banks.

Artificial intelligence's ability to find previously unknown ways to hack into computer systems is in the global spotlight, as leading labs release ever-more advanced models.

Artex was "abused by some bad actors" to launch cyberattacks, the tool's developer "Autumn-27" wrote Thursday on code-hosting platform GitHub.

"Given the misuse of the tool, the Artex project will no longer be updated and will be converted to closed-source," the developer said.

"No further versions will be released to the public, nor will maintenance support be provided."

Artex had been open-source -- allowing programmers to download its underlying code and customize it to suit their purposes.

Using Artex for cyberattacks was "entirely contrary to" the developer's intentions, they added, without referring directly to the alleged South Korean cases.

South Korea's Financial Services Commission says more than 68,000 people have been affected by the hacks.

Shinhan Bank, one of the breached institutions, said information attached to loan applications for about 25,000 customers had been leaked -- including names, phone numbers and annual income.

- 'Financially motivated' -

In Japan, meanwhile, around 20 companies have said their data may have been compromised in a spate of similar cyberattacks potentially impacting millions of customers.

The Japanese government has called on companies to strengthen their cyber defenses.

"At this stage, it is not clear what the background to these cases is or whether there are any links between them", Japan police chief Yoshinobu Kusunoki said on Thursday.

As AI makes hacking more sophisticated, "the scope of the damage is spreading across all areas on a scale that is difficult to compare with the past", South Korean President Lee Jae Myung said on Tuesday.

US cybersecurity giant CrowdStrike said on Wednesday that its investigations into digital clues suggested the attacker had used Artex, and was potentially a 26-year-old based in China.

"The threat actor is likely a Chinese speaker and financially motivated," a CrowdStrike blog post said.

Experts told AFP that Artex going closed-source would make the code harder for new users to obtain and adapt.

But the decision "cannot remove copies already downloaded or prevent people from continuing to use them", said Poe Zhao, founder of the analysis publication Hello China Tech.

"Because the code was public, people could download it, change it and run it themselves. The developer could ask users to follow the rules, but had little control over their actions," he added.

Ilya Kulyatin, CEO of Foundry Labs and founder of the Tokyo AI (TAI) tech community, said users of open-source programs can "remove restrictions built into the tool" which "makes certain forms of misuse easier".

"But openness also benefits defenders: researchers can inspect the code, identify weaknesses and improve protection."