عربي English Türkçe اردو فارسى
Logo
Latest News
Technology

AI Experts Ready ‘Humanity’s Last Exam’ to Stump Powerful Tech

Figurines with computers and smartphones are seen in front of the words "Artificial Intelligence AI" in this illustration taken, February 19, 2024. (Reuters)
Figurines with computers and smartphones are seen in front of the words "Artificial Intelligence AI" in this illustration taken, February 19, 2024. (Reuters)
TT
TT

AI Experts Ready ‘Humanity’s Last Exam’ to Stump Powerful Tech

Figurines with computers and smartphones are seen in front of the words "Artificial Intelligence AI" in this illustration taken, February 19, 2024. (Reuters)
Figurines with computers and smartphones are seen in front of the words "Artificial Intelligence AI" in this illustration taken, February 19, 2024. (Reuters)

A team of technology experts issued a global call on Monday seeking the toughest questions to pose to artificial intelligence systems, which increasingly have handled popular benchmark tests like child's play.

Dubbed "Humanity's Last Exam," the project seeks to determine when expert-level AI has arrived. It aims to stay relevant even as capabilities advance in future years, according to the organizers, a non-profit called the Center for AI Safety (CAIS) and the startup Scale AI.

The call comes days after the maker of ChatGPT previewed a new model, known as OpenAI o1, which "destroyed the most popular reasoning benchmarks," said Dan Hendrycks, executive director of CAIS and an advisor to Elon Musk's xAI startup.

Hendrycks co-authored two 2021 papers that proposed tests of AI systems that are now widely used, one quizzing them on undergraduate-level knowledge of topics like US history, the other probing models' ability to reason through competition-level math. The undergraduate-style test has more downloads from the online AI hub Hugging Face than any such dataset.

At the time of those papers, AI was giving almost random answers to questions on the exams. "They're now crushed," Hendrycks told Reuters.

As one example, the Claude models from the AI lab Anthropic have gone from scoring about 77% on the undergraduate-level test in 2023, to nearly 89% a year later, according to a prominent capabilities leaderboard.

These common benchmarks have less meaning as a result.

AI has appeared to score poorly on lesser-used tests involving plan formulation and visual pattern-recognition puzzles, according to Stanford University’s AI Index Report from April. OpenAI o1 scored around 21% on one version of the pattern-recognition ARC-AGI test, for instance, the ARC organizers said on Friday.

Some AI researchers argue that results like this show planning and abstract reasoning to be better measures of intelligence, though Hendrycks said the visual aspect of ARC makes it less suited to assessing language models. "Humanity’s Last Exam" will require abstract reasoning, he said.

Answers from common benchmarks may also have ended up in data used to train AI systems, industry observers have said. Hendrycks said some questions on "Humanity's Last Exam" will remain private to make sure AI systems' answers are not from memorization.

The exam will include at least 1,000 crowd-sourced questions due November 1 that are hard for non-experts to answer. These will undergo peer review, with winning submissions offered co-authorship and up to $5,000 prizes sponsored by Scale AI.

"We desperately need harder tests for expert-level models to measure the rapid progress of AI," said Alexandr Wang, Scale's CEO.

One restriction: the organizers want no questions about weapons, which some say would be too dangerous for AI to study.

Most Viewed

 
 
Technology

AI to Track Icebergs Adrift at Sea in Boon for Science

© Jonathan NACKSTRAND / AFP
© Jonathan NACKSTRAND / AFP
TT
TT

AI to Track Icebergs Adrift at Sea in Boon for Science

© Jonathan NACKSTRAND / AFP
© Jonathan NACKSTRAND / AFP

British scientists said Thursday that a world-first AI tool to catalogue and track icebergs as they break apart into smaller chunks could fill a "major blind spot" in predicting climate change.

Icebergs release enormous volumes of freshwater when they melt on the open water, affecting global climate patterns and altering ocean currents and ecosystems, reported AFP.

But scientists have long struggled to keep track of these floating behemoths once they break into thousands of smaller chunks, their fate and impact on the climate largely lost to the seas.

To fill in the gap, the British Antarctic Survey has developed an AI system that automatically identifies and names individual icebergs at birth and tracks their sometimes decades-long journey to a watery grave.

Using satellite images, the tool captures the distinct shape of icebergs as they break off -- or calve -- from glaciers and ice sheets on land.

As they disintegrate over time, the machine performs a giant puzzle problem, linking the smaller "child" fragments back to the "parent" and creating detailed family trees never before possible at this scale.

It represents a huge improvement on existing methods, where scientists pore over satellite images to visually identify and track only the largest icebergs one by one.

The AI system, which was tested using satellite observations over Greenland, provides "vital new information" for scientists and improves predictions about the future climate, said the British Antarctic Survey.

Knowing where these giant slabs of freshwater were melting into the ocean was especially crucial with ice loss expected to increase in a warming world, it added.

"What's exciting is that this finally gives us the observations we've been missing," Ben Evans, a machine learning expert at the British Antarctic Survey, said in a statement.

"We've gone from tracking a few famous icebergs to building full family trees. For the first time, we can see where each fragment came from, where it goes and why that matters for the climate."

This use of AI could also be adapted to aid safe passage for navigators through treacherous polar regions littered by icebergs.

Iceberg calving is a natural process. But scientists say the rate at which they were being lost from Antarctica is increasing, probably because of human-induced climate change.

 

Technology

AMD Predicts Weaker First-Quarter Sales, Shares Plunge on Nvidia Comparisons

An AMD logo and a computer motherboard appear in this illustration created on August 25, 2025. (Reuters)
An AMD logo and a computer motherboard appear in this illustration created on August 25, 2025. (Reuters)
TT
TT

AMD Predicts Weaker First-Quarter Sales, Shares Plunge on Nvidia Comparisons

An AMD logo and a computer motherboard appear in this illustration created on August 25, 2025. (Reuters)
An AMD logo and a computer motherboard appear in this illustration created on August 25, 2025. (Reuters)

Advanced Micro Devices on Tuesday forecast a slight decline in quarterly revenue, raising concerns about whether it ​can effectively challenge Nvidia in the booming AI market and sending its shares tumbling 8% in after-hours trade.

The lackluster prediction comes despite an unexpected boost from sales of certain artificial intelligence chips to China, which began in the last quarter after the Trump administration approved a license for orders that AMD received in early 2025.

And without those sales to China which generated $390 million, AMD's data-center segment would have missed estimates for the fourth quarter.

AMD said it expects revenue of about $9.8 billion this quarter, plus or minus $300 million. That's down from $10.27 billion in the fourth-quarter which was up 34% year-on-year and ahead of LSEG ‌estimates for $9.67 billion.

PALES ‌NEXT TO NVIDIA

Though AMD is seen as one of the ‌few ⁠contenders ​that can seriously ‌challenge Nvidia, investors noted the stark contrast between the two companies' performances. AMD expects an adjusted gross margin of 55% this quarter. Nvidia has said it expects adjusted gross margin in the mid-70% range during its fiscal 2027.

"The expectations for large blowout quarters for AI-related hardware companies have skewed what the market is looking for," said Bob O'Donnell, president of TECHnalysis Research.

The forecast for the current first quarter includes $100 million from sales to China, where the situation remains "dynamic," AMD CEO Lisa Su said on a conference call with investors.

The US government ⁠has placed restrictions on the exports of advanced chips to China, but AMD received licenses to sell modified versions of its MI300 series ‌of AI chips there. Its MI308 chip competes with Nvidia's H20 ‍chip in China.

OPENAI SALES

AMD has accelerated its ‍product launches and is moving into selling full AI systems to better compete against Nvidia, which now ‍provides "rack-scale" systems that combine GPUs, CPUs and networking gear.

Last year, it entered into a multi-year deal to supply AI chips to ChatGPT-owner OpenAI, which would bring in tens of billions of dollars in annual revenue and give the startup the option to buy up to roughly 10% of the chipmaker.

Su reiterated on Tuesday that the company ​expects sales of a new flagship AI server to OpenAI and others to rise rapidly in the second half of this year, saying a global memory-chip crunch will not ⁠slow its plans.

"I do not believe that we will be supply-limited in terms of the ramp that we put in place," Su said.

BEYOND OPENAI

As Big Tech and governments across the globe double down on investing in AI hardware, shares in Santa Clara, California-based AMD have doubled since the start of 2025, outperforming a 60% bump in the broader chip index.

But analysts remain concerned that AMD's success remains tied to a handful of customers that rivals such as Nvidia could try to poach. Reuters reported this week that Nvidia made a $20 billion move to hire most of chip startup Groq's founders after OpenAI held chip supply discussions with the startup.

"Growth appears concentrated in large deployments and specific regions, and China shipments are significant enough to influence a quarter," said eMarketer analyst Gadjo Sevilla.

Revenue in AMD's key data-center segment grew 39% to $5.38 billion in the ‌fourth quarter. But excluding sales of the MI308, which is a data-center chip, that revenue would have been $4.99 billion, below estimates of $5.07 billion.

Technology

Switch 2 Sales Boost Nintendo Results but Chip Shortage Looms

This photo taken on November 4, 2025 shows a woman taking photos of a Super Mario figure at the Nintendo Tokyo store in Tokyo. (AFP)
This photo taken on November 4, 2025 shows a woman taking photos of a Super Mario figure at the Nintendo Tokyo store in Tokyo. (AFP)
TT
TT

Switch 2 Sales Boost Nintendo Results but Chip Shortage Looms

This photo taken on November 4, 2025 shows a woman taking photos of a Super Mario figure at the Nintendo Tokyo store in Tokyo. (AFP)
This photo taken on November 4, 2025 shows a woman taking photos of a Super Mario figure at the Nintendo Tokyo store in Tokyo. (AFP)

The runaway success of the Switch 2 console drove up Nintendo's net profit by more than 50 percent in the nine months to December, the Japanese video game giant said Tuesday.

But a global memory chip shortage, created by frenzied demand for artificial intelligence hardware, could push up manufacturing costs.

The Switch 2 became the world's fastest-selling games console after launching to a fan frenzy last summer.

It is the successor to the original Switch, which soared in popularity during the pandemic when games such as "Animal Crossing" struck a chord during long lockdowns.

Both are hybrid devices that can be connected to a TV or used on-the-go.

In April-December, net profit jumped 51.3 percent year-on-year to 358.9 billion yen ($2.3 billion), and revenue nearly doubled on-year to 1.9 trillion yen, Nintendo said.

But the firm kept its annual unit sales target for the Switch 2 steady at 19 million, and also held its full-year net profit forecast of 350 billion yen.

"Nintendo Switch 2 got off to a good start following its launch on June 5 and unit sales continued to grow through the holiday season," the company said.

Nearly 17.4 million Switch 2 devices were sold in the nine-month period, it added.

"Maintaining momentum is certainly a big focus for Nintendo," Krysta Yang of the Nintendo-focused Kit and Krysta Podcast told AFP.

A lack of heavy-hitting first-party new games for the Switch 2 in coming months risks hindering growth, although third-party titles such as "Resident Evil Requiem" should help fill the gap, she said.

Nintendo said Tuesday it planned to release "Mario Tennis Fever" this month and "Pokemon Pokopia" in March.

While the firm is diversifying into hit movies and theme parks, consoles remain the core of its business.

The Switch 1 has now sold 155.37 million units -- overtaking the Nintendo DS console to be its best-selling hardware of all time.

But soaring prices for memory chips, used in gaming consoles as well as phones, laptops and other electronics, will likely be a headwind for the company.

Their prices have been pushed up as chipmakers focus on producing the advanced memory chips in huge demand to power AI data centers.

"Nintendo and other console manufacturers are publicly keeping quiet about the impact of the shortage," gaming industry consultant Serkan Toto told AFP.

But "users can forget the past when consoles always became cheaper in tandem with component costs falling over time", with price hikes potentially on the way in 2026, he said.

Yang said she thought a price increase for the Switch 2 "is not out of the question" but added that Nintendo "would likely exhaust all other options" before doing so.