- Kimsuky used local AI tools to evade monitoring and enhance operations
- Researchers observed extensive AI-driven capability building across the group’s infrastructure
- Defenders urged behavior-based detection to spot evolving AI-enabled threats
North Korean hackers have found a way to use Generative Artificial Intelligence (GenAI) to supercharge their activities without tipping off the tool’s maintainers.
When people use AI tools like ChatGPT or Claude, their activities can be (at least to some extent) tracked and curbed - with OpenAI recently identifying and terminating multiple ChatGPT accounts used in phishing and human trafficking.
That is why Kimsuky - a known state-sponsored North Korean threat actor, used Ollama, GPT4All and Msty locally, allowing them to process documents without sending any sensitive information to outside AI services.
"Consistent process of capability development"
The attacks were spotted by security researchers Genians who “conducted months of tracking and log analysis on the infrastructure utilized as C2 in this campaign,” to identify the tools they used.
Aside from the three LLMs, they also used retrieval augmented generation (RAG) tools for document search, as well as AI agent development frameworks, text-to-speech software, and an AI-assisted coding tool called Cursor.
Using AI to write malicious code is not as simple as it sounds, due to various guardrails set up by the developers. As a result, AI in crime has been mostly limited to drafting phishing emails and crafting authentic-looking but malicious landing pages. However, Kimsuky has shown that AI in cybercrime continues to evolve and is becoming an ever-greater threat.
“What was observed in the threat actor's infrastructure was not merely evidence of several documents being created with AI, but a consistent process of capability development: establishing local LLM runtime environments, configuring RAG based on documents in the actor's possession, collecting AI agent development frameworks, and acquiring libraries for integration with external commercial AI services,” Genians concluded.
As a result, defenders must move from content-based assessment to behavior-based detection, the researchers warned, saying this should serve “as the fundamental premise of security recommendations.”
“In addition to indicator of compromise (IoC)-based detection, organizations should contextually correlate the sequence of anomalous activities following LNK execution, including PowerShell execution, persistence establishment, and external communications, to assess the overall threat level.”
