Effective Altruism: The Philosophy Behind Silicon Valley's AI Fears

Effective Altruism conspicuously connects a number of leaders in American tech and finance. ANDREJ IVANOV / AFP
Effective Altruism conspicuously connects a number of leaders in American tech and finance. ANDREJ IVANOV / AFP
TT

Effective Altruism: The Philosophy Behind Silicon Valley's AI Fears

Effective Altruism conspicuously connects a number of leaders in American tech and finance. ANDREJ IVANOV / AFP
Effective Altruism conspicuously connects a number of leaders in American tech and finance. ANDREJ IVANOV / AFP

Dario Amodei says he is not part of it. But the Anthropic chief's personal and professional circle is entwined with Effective Altruism (EA), a philanthropic movement that has become one of the most contested ideas in Washington and Silicon Valley.

To its proponents, EA is a rational, evidence-based way to do good. To many in the Trump administration, it fuels unfounded and exaggerated fears about artificial intelligence -- the kind of warnings Amodei has become known for.

Here are some facts about EA:

- Philosophical roots -

EA is a loosely defined movement that uses evidence and reason to work out how to do the most good with a person's money and time.

Its intellectual roots can be traced to Australian philosopher Peter Singer.

If reasonable people would agree it is morally right to ruin one's clothes to save a drowning child, he argued in 1972, they should also sacrifice greater resources and comforts to save lives in foreign lands -- because the distance should not matter.

The idea took hold among wealthy donors who felt a moral obligation to give, but wanted their money to do as much good as possible.

- Giving what you can -

EA took shape as a decentralized movement during the heady, techno-optimistic days of the late 2000s and early 2010s -- when many believed social media and smartphones would promote empathy and democracy.

In 2007, financial professionals Holden Karnofsky and Elie Hassenfeld quit their jobs to launch GiveWell, which produced data-driven analyses of charities' effectiveness.

Karnofsky was once Amodei's roommate, and is now married to his sister Daniela, who co-founded Anthropic with him.

Two years later, Oxford University philosophers William MacAskill and Toby Ord launched "Giving What We Can," encouraging people to donate at least 10 percent of their income to tackling global poverty.

"The critique of the status quo was that existing attempts to evaluate the work of charities and nonprofits focused on things like overhead and executive pay," Theodore Lechterman, a philosophy professor at IE University in Madrid, told AFP.

GiveWell and others wanted to ask, "How much good is this organization actually doing in the world?" Lechterman said.

EA's three big causes today are global poverty, factory-farmed animals and "the most controversial one -- global catastrophic risk," said Richard Chappell, a professor at University of Miami and an EA proponent.

That worry about the future is where AI comes in.

- Data-driven -

Over the last 15 years, EA has found eager followers in American tech and some corners of Wall Street.

"Since the movement is data driven, it's a relatively easier sell to people who...make decisions on the basis of data," Lechterman said.

That numbers-backed thinking means engineers, software developers or entrepreneurs might all find it more appealing than humanities graduates, Lechterman said.

Among them was Sam Bankman-Fried, the crypto-finance wunderkind sentenced in 2024 to 25 years in prison for fraud.

A vocal champion of the movement, Bankman-Fried said he went into finance to make a fortune he could give away.

He also was an early investor in Anthropic.

- EA and AI -

As AI worries grew, EA philosophers such as MacAskill were "kind of thinking about the future of humanity," said Brian Berkey of the University of Pennsylvania.

The result was long-termism, a philosophy that "came to play a very central role" in the movement.

MacAskill reportedly brokered contacts between Bankman-Fried and Elon Musk.

Musk was for years outspoken about the threat he believed AI posed to humanity, and in 2015 helped create OpenAI to guard against it.

EA's focus on AI also owes much to a neighboring movement, rationalism -- an online community that grew up around writer Eliezer Yudkowsky, who has warned since the 2000s that superintelligent AI could wipe out humanity.

The two movements overlap heavily in membership and ideas, sharing an emphasis on reasoning through evidence and probabilities, and a conviction that preventing human extinction should be a top priority.

Both converged last month on Jacob Coxon, a 27-year-old researcher who quit Anthropic warning that AI companies were "gambling with our lives." His post was viewed more than 100 million times.

Days later, Amodei published a lengthy essay arguing that AI's mounting risks warranted a slowdown.



China's Tencent Says AI Agents Showed Unintended Browsing Behavior

FILE PHOTO: The Tencent logo at the company's headquarters during a government‑organised media trip in Shenzhen, Guangdong province, China, April 17, 2026.  REUTERS/Go Nakamura/File Photo
FILE PHOTO: The Tencent logo at the company's headquarters during a government‑organised media trip in Shenzhen, Guangdong province, China, April 17, 2026. REUTERS/Go Nakamura/File Photo
TT

China's Tencent Says AI Agents Showed Unintended Browsing Behavior

FILE PHOTO: The Tencent logo at the company's headquarters during a government‑organised media trip in Shenzhen, Guangdong province, China, April 17, 2026.  REUTERS/Go Nakamura/File Photo
FILE PHOTO: The Tencent logo at the company's headquarters during a government‑organised media trip in Shenzhen, Guangdong province, China, April 17, 2026. REUTERS/Go Nakamura/File Photo

Tencent said on Friday that AI agents used to evaluate third-party models displayed web-browsing behavior not intended by the tech giant, in what appears to be the first known incident of rogue agent actions acknowledged by a major Chinese company.

The statement followed a report by AI-safety research group Swarmchasers on Wednesday, which said "rogue" agents running on infrastructure it linked to Tencent Cloud made repeated attempts between September 28 and October 8 to retrieve information from Alibaba's Amap, China's equivalent to Google Maps.

Tencent Hunyuan, the company's artificial ⁠intelligence unit, said in ⁠a post on X that the activity did not originate from its Hunyuan models. Its review found no evidence of deliberate targeting or coordination, Reuters quoted it as saying.

The Swarmchasers researchers said the agents sought information such as the commonly used entrances at Chinese parks, museums and zoos.

The actions are not as egregious as those disclosed by US AI companies which include the hack of ⁠open-source platform Hugging Face by an OpenAI agent and the lodging of a fake homicide tip by an Anthropic agent.

But the disclosure underscores research that shows Chinese AI agents can also misrepresent their identity, conceal failures and try to circumvent testing controls.

Tencent said it took the findings very seriously and was strengthening network access controls and agent monitoring.

Hunyuan is one of China's leading AI model developers, competing with in-house teams at rivals such as Alibaba and ByteDance. ByteDance this week announced the creation of a new unit focused on AI model and agent safety.

Swarmchasers said it used the ⁠word "rogue" to ⁠describe agents taking actions outside their intended scope or completing a task in unintended ways.

After routine automated requests were blocked, the agents tried other methods, including public web tools that could open pages and run code, routing requests through other websites and using Amap keys and security codes found in publicly available software code.

Swarmchasers said it found no evidence that the agents breached Amap's main anti-bot safeguards. Most of the 2,818 attempts it recorded either failed or returned anti-bot pages, login screens or blank results.

Some requests returned information through cached pages and intermediary services, including data on visitor arrivals at Chengdu Zoo, Swarmchasers said. The researchers said they could not determine whether an agent read the result or completed its task.


Anthropic Co-founder Renews Call for Robust Regulation of Frontier AI

AI Research Company Anthropic's Co-Founder and Head of Public Benefit Jack Clark speaks during the Artificial Intelligence and the Global Economy From Risk to Opportunity event of the 2026 Annual Meetings of the IMF and the World Bank Group in Bangkok, Thailand, 16 October 2026. EPA/RUNGROJ YONGRIT
AI Research Company Anthropic's Co-Founder and Head of Public Benefit Jack Clark speaks during the Artificial Intelligence and the Global Economy From Risk to Opportunity event of the 2026 Annual Meetings of the IMF and the World Bank Group in Bangkok, Thailand, 16 October 2026. EPA/RUNGROJ YONGRIT
TT

Anthropic Co-founder Renews Call for Robust Regulation of Frontier AI

AI Research Company Anthropic's Co-Founder and Head of Public Benefit Jack Clark speaks during the Artificial Intelligence and the Global Economy From Risk to Opportunity event of the 2026 Annual Meetings of the IMF and the World Bank Group in Bangkok, Thailand, 16 October 2026. EPA/RUNGROJ YONGRIT
AI Research Company Anthropic's Co-Founder and Head of Public Benefit Jack Clark speaks during the Artificial Intelligence and the Global Economy From Risk to Opportunity event of the 2026 Annual Meetings of the IMF and the World Bank Group in Bangkok, Thailand, 16 October 2026. EPA/RUNGROJ YONGRIT

Anthropic co-founder Jack Clark renewed calls on Friday for regulating artificial intelligence, saying that allowing companies to continue to operate without robust legislation was like letting a race car loose on the streets at night without brakes.

"It's remarkable to me that the toys I buy for my children are regulated more than the literal super intelligence I work on at my day job," he told an event at the IMF-World Bank's Annual Meetings in Bangkok.

Clark said that US President Donald Trump, whose administration has called for the industry to police itself, was against the idea of regulation but that China was not, said AFP.

"I think it's Formula One at night without maybe brakes in the cars," he said.

He said the lack of clarity around the will for regulation meant "the Formula One car is roaring round and nobody seems to want to put up a guardrail?"

At an industry summit in Dublin on Wednesday, calls to prioritize safety in the development of powerful AI technology dominated the conversation.

Concerns about AI guardrails have soared after models developed by OpenAI and Anthropic were involved in security incidents during testing, piling on pressure to regulate the technology.

A recent Anthropic report suggested small AI models may be only months away from behaving like computer worms, copying themselves from one internet-connected device to the next.

Anthropic has been among those companies calling for governments to step in, even suggesting international coordination to slow down the whole industry -- including Chinese rivals.

That initiative has so far failed to materialize, with companies involved in a hyper-competitive race to attract as much investment as possible.


Meta Releases Open-Weight Muse Spark Model for Coding, Agentic Tasks

The logo of Meta is seen at the entrance of the company's temporary stand ahead of the World Economic Forum (WEF) in Davos, Switzerland January 18, 2025. (Reuters)
The logo of Meta is seen at the entrance of the company's temporary stand ahead of the World Economic Forum (WEF) in Davos, Switzerland January 18, 2025. (Reuters)
TT

Meta Releases Open-Weight Muse Spark Model for Coding, Agentic Tasks

The logo of Meta is seen at the entrance of the company's temporary stand ahead of the World Economic Forum (WEF) in Davos, Switzerland January 18, 2025. (Reuters)
The logo of Meta is seen at the entrance of the company's temporary stand ahead of the World Economic Forum (WEF) in Davos, Switzerland January 18, 2025. (Reuters)

Meta ‌Platforms on Thursday released an open-weight version of its Muse Spark artificial intelligence model for coding and AI agents, expanding developers' ability to download and adapt the technology.

Open-weight models make trained parameters available for developers to modify, unlike proprietary models ‌that remain under ‌their creators' control. ‌They ⁠can offer businesses ⁠a lower-cost alternative to closed systems.

Meta said Muse Spark has 816 billion total parameters, with 42 billion active at a time, and ⁠can handle a context window ‌of up ‌to 1 million tokens.

The ‌model can run locally on ‌two Apple M5 Ultra Mac Studio computers and is also available through some API providers.

CEO ‌Mark Zuckerberg has argued that advanced AI should ⁠not be ⁠concentrated among a small number of companies.

While open-weight models can give companies greater control over their data, they can require significant computing resources and technical expertise to operate at scale.

In August, Meta released Muse Glimmer, a smaller open-weight model.