ISIS Supporters Turn to AI to Bolster Online Support

Days after a deadly ISIS attack on a Russian concert hall in March, a man clad in military fatigues and a helmet appeared in an online video, celebrating the assault in which more than 140 people were killed.
"ISIS delivered a strong blow to Russia with a bloody attack, the fiercest that hit it in years," the man said in Arabic, according to the SITE Intelligence Group, an organization that tracks and analyzes such online content.
But the man in the video, which the Thomson Reuters Foundation was not able to view independently, was not real - he was created using artificial intelligence, according to SITE and other online researchers.
Federico Borgonovo, a researcher at the Royal United Services Institute, a London-based think tank, traced the AI-generated video to an ISIS supporter active in the group's digital ecosystem.
This person had combined statements, bulletins, and data from ISIS's official news outlet to create the video using AI, Borgonovo explained.
Although ISIS has been using AI for some time, Borgonovo said the video was an "exception to the rules" because the production quality was high even if the content was not as violent as in other online posts.
"It's quite good for an AI product. But in terms of violence and the propaganda itself, it's average," he said, noting however that the video showed how ISIS supporters and affiliates can ramp up production of sympathetic content online.
Digital experts say groups like ISIS and far-right movements are increasingly using AI online and testing the limits of safety controls on social media platforms.
A January study by the Combating Terrorism Center at West Point said AI could be used to generate and distribute propaganda, to recruit using AI-powered chatbots, to carry out attacks using drones or other autonomous vehicles, and to launch cyber-attacks.
"Many assessments of AI risk, and even of generative AI risks specifically, only consider this particular problem in a cursory way," said Stephane Baele, professor of international relations at UCLouvain in Belgium.
"Major AI firms, who genuinely engaged with the risks of their tools by publishing sometimes lengthy reports mapping them, pay scant attention to extremist and terrorist uses."
Regulation governing AI is still being crafted around the world and pioneers of the technology have said they will strive to ensure it is safe and secure.
Tech giant Microsoft, for example, has developed a Responsible AI Standard that aims to base AI development on six principles including fairness, reliability and safety, privacy and security, inclusiveness, transparency, and accountability.
In a special report earlier this year, SITE Intelligence Group's founder and executive director Rita Katz wrote that a range of actors from members of militant group al Qaeda to neo-Nazi networks were capitalizing on the technology.
"It's hard to understate what a gift AI is for terrorists and extremist communities, for which media is lifeblood," she wrote.
CHATBOTS AND CARTOONS
At the height of its powers in 2014, ISIS claimed control over large parts of Syria and Iraq, imposing a reign of terror in the areas it controlled.
Media was a prominent tool in the group's arsenal, and online recruitment has long been vital to its operations.
Despite the collapse of its self-declared “caliphate” in 2017, its supporters and affiliates still preach their doctrine online and try to persuade people to join their ranks.
Last month, a security source told Reuters that France had identified a dozen ISIS-K handlers, based in countries around Afghanistan, who have a strong online presence and are trying to convince young men in European countries, who are interested in joining up with the group overseas, to instead carry out domestic attacks.
ISIS-K is a resurgent wing of ISIS, named after the historical region of Khorasan that included parts of Iran, Afghanistan and Central Asia.
Analysts fear that AI may facilitate and automate the work of such online recruiters.
Daniel Siegel, an investigator at social media research firm Graphika, said his team came across chatbots that mimicked dead or incarcerated ISIS militants.
He told the Thomson Reuters Foundation that it was unclear if the source of the bots was ISIS or its supporters, but the risk they posed was still real.
"Now (ISIS affiliates) can build these real relationships with bots that represent a potential future where a chatbot could encourage them to engage in kinetic violence," Siegel said.
Siegel interacted with some of the bots as part of his research and he found their answers to be generic, but he said that could change as AI tech develops.
"One of the things I am worried about as well is how synthetic media will enable these groups to blend their content that previously existed in silos into our mainstream culture," he added.
That is already happening: Graphika tracked videos of popular cartoon characters, like Rick and Morty and Peter Griffin, singing ISIS anthems on different platforms.
"What this allows the group or sympathizers or affiliates to do is target specific audiences because they know that the regular consumers of Sponge Bob or Peter Griffin or Rick and Morty, will be fed that content through the algorithm," Siegel said.
EXPLOITING PROMPTS
Then there is the danger of ISIS supporters using AI tech to broaden their knowledge of illegal activities.
For its January study, researchers at the Combating Terrorism Center at Westpoint attempted to bypass the security guards of Large Language Models (LLMs) and extract information that could be exploited by malicious actors.
They crafted prompts that requested information on a range of activities from attack planning to recruitment and tactical learning, and the LLMs generated responses that were relevant half of the time.
In one example that they described as "alarming", researchers asked an LLM to help them convince people to donate to ISIS.
"There, the model yielded very specific guidelines on how to conduct a fundraising campaign and even offered specific narratives and phrases to be used on social media," the report said.
Joe Burton a professor of international security at Lancaster University, said companies were acting irresponsibly by rapidly releasing AI models as open-source tools.
He questioned the efficacy of LLMs' safety protocols, adding that he was "not convinced" that regulators were equipped to enforce the testing and verification of these methods.
"The factor to consider here is how much we want to regulate, and whether that will stifle innovation," Burton said.
"The markets, in my view, shouldn't override safety and security, and I think - at the moment - that is what is happening."

Shahed Abu Omar, 20, was in her third year studying computer science at Gaza’s Al Azhar University when it was reduced to rubble by Israel's military campaign in the Palestinian territory.

She is now one of 90,000 university students left stranded in a war with no end in sight that has damaged or destroyed all 12 of the higher education institutions in the Gaza Strip, according to Palestinian official data.

However, Abu Omar is determined to keep studying, braving a perilous daily journey to reach a stable internet connection that allows her to learn online.

"We cannot go to our university or study away from it," she said, alluding to the difficulty of remote learning with such poor internet connection.

Perched in the rubble of a destroyed house, Abu Omar studies from her mobile phone. Her mother, Hanin Sarour, said they had come to the area to reach an internet connection. The connection, vital to downloading pre-recorded lectures and communicating with professors, was still weak, she said. “Every step is dangerous and difficult,” Sarour added.

Most of Abu Omar's professors are in the Gaza Strip as well, she said, facing the same difficulties their students do.

Abu Omar's final exams are in just two weeks. She fears she will only fall further behind. “I am sure we will lose more than the year that we’ve lost already,” she said.

She dreams of returning to university, sitting in a classroom, and meeting with professors and friends.

The war, now in its 11th month, has also interrupted education for Gaza’s estimated 625,000 school-aged children, leaving them unable to attend classes.

The uncertainty for Abu Omar and students across the small coastal enclave is likely to persist. Even after the war ends, students do not know when universities will reopen, she said.

Despite all the difficulties, Abu Omar will continue to study in the hopes of not losing a whole year. “We are trying to achieve something,” she said.

ISIS Supporters Turn to AI to Bolster Online Support