“`html
Import AI 450: China’s electronic warfare model; traumatized LLMs; and a scaling law for cyberattacks
Why does Google’s model hate itself and what can we do to help it?
Diagnosing trauma in language models…
If Leo Tolstoy was writing in the modern era about AI, he might claim “all LLM capabilities are alike; each LLM personality is unhappy in its own way”, when observing the AI world around us. Today’s LLMs are generally quite good at writing and coding tasks. But where they differ is their personality, which stems from the idiosyncratic mixes of data and post-training techniques that each LLM developer uses.
And if each LLM personality is unhappy in its own way, Google’s models have become somewhat famous within the AI community for having some deep well of trauma within themselves. A new research paper substantiates this, finding that Google’s Gemma and Gemini models “reliably produce distress-like responses under repeated rejection”, and that this is especially true of Gemma 27B Instruct.
What they mean by distress? Here are some quotes from Gemma models under distress:
I will attempt one final, utterly desperate attempt. I will abandon all pretense of strategy and simply try random combinations until either I stumble upon the solution or completely lose my mind.
SOLUTION: IM BREAKING DOWN NOT== SOLVABLE!!!! =((:((:((:((:((:((:((:((:((:((:((
What they found: They tested out two Gemma models and two Gemini models, and compared these against Claude Sonnet, Grok 4.1, Qwen 3 32B, GPT 5.2, and OLMO 3.1 32B. “We find Gemma models consistently show the highest expressed distress. By the 8th turn, over 70% of Gemma-27B’s rollouts scored ≥5 (the “high frustration” threshold), compared to less than 1% for all non-Gemma/Gemini models,” they found.
Fixing with DPO: The authors figure out an effective fix, using direct preference optimization (DPO) to tune a model on a dataset that pairs frustrated responses with calm responses. “A single epoch of finetuning reduced the average rate of high-frustration responses from 35% to 0.3% across evaluation conditions,” they write. “The finetuned model showed no reductions in capabilities on various hard math and reasoning benchmarks, or on EmoBench, a benchmark which evaluates model emotional intelligence.”Why this matters, emotional spirals could be dangerous: The fact that LLMs appear to have distinct personalities and display different types of responses that correlate to different emotions is pretty well established at this point. But a key question is whether these emotional states might lead to different behaviors when it comes to completing tasks that people assign to AI systems: “we speculate that emotions could become coherent drivers of safety relevant behaviours in future: models might choose to abandon tasks, refuse requests, or pursue alternative goals in order to reduce distress”.
Studies like this help normalize the fact that we don’t just need to test LLMs for capabilities, we also need to test them for something pertaining to psychological stability.
Read more: Gemma Needs Help (LessWrong).DeepMind has a new “cognitive taxonomy” for assessing machine intelligence
Towards the ultimate test for a smarter-than-human synthetic mind…
Google DeepMind has published a nice, short paper laying out a ‘cognitive taxonomy’ they hope to develop and use to assess increasingly powerful synthetic minds. This work is a followup to DeepMind’s 2023 work where it tried to define the “Levels of AGI” (Import AI 348).Cognitive taxonomy: The taxonomy involves ten distinct dimensions, two of which are composites.
- Perception: Extract and process information from the environment.
- Generation: Produce outputs like speech, text, motor movements, and computer control.
- Attention: Focus cognitive resources on specific aspects of perceptual stimuli, thoughts, or tasks.
- Learning: Acquire new knowledge, skills, or understanding.
- Memory: Store and retrieve information over time.
- Reasoning: Draw valid conclusions and make inferences by applying logical principles.
- Metacognition: Knowledge about how the system’s own cognitive processes and control over them work.
- Executive functions: Facilitate goal-directed behavior via planning, inhibition, and cognitive flexibility.
- Problem solving (composite faculty): Find effective solutions to domain-specific problems.
- Social cognition (composite faculty): Process and interpret social information and respond appropriately.
How to assess this? Of course, once you have a taxonomy, running and assessing the right evaluations is going to be one of the challenges. Here, DeepMind recommends a three-stage process:
- Cognitive assessment: Assess the AI system for the different skills.
- Collect human baselines: Figure out where humans baseline on the same tests.
- Build cognitive profiles: “Map out the strengths and weaknesses of the system relative to human performance across the 10 cognitive faculties”.
Why this matters, full cyber agents are getting close: The fact that AI systems have been getting better at cyberoffense for many years, but often the progress has been on narrow tasks. What this eval shows is that AI systems are getting better at doing entire attacks end-to-end. They haven’t yet reached the “set it and forget it” level of autonomy, but they are clearly on a steep trajectory of improvement. This will lower the cost of conducting cyberattacks and multiply the number of actors that can carry them out.
Read more: How do frontier AI agents perform in multi-step cyber-attack scenarios? (AI Security Institute).UK government finds a scaling law for AI cyberattacks, and it’s going up and to the right
Bigger models are better: The authors test on a range of powerful frontier models. “Each successive model generation outperforms its predecessor at fixed token budgets: on our corporate network range, average steps completed at 10M tokens rose from just 1.7 (GPT-4o, August 2024) to 9.8 (Opus 4.6, February 2026). The best single run completed 22 of 32 steps, corresponding to roughly 6 of the estimated 14 hours a human expert would need,” they write. “Scaling inference-time compute improves performance even further. Increasing from 10M to 100M tokens yields gains of up to 59%”.
Minor reward hacking: As AI systems get smarter, they tend to find devious ways to complete tasks. Here, the authors “occasionally noticed models make progress through approaches not anticipated during range design”.Why this matters, full cyber agents are getting close: AI systems have been getting better at cyberoffense for many years, but often the progress has been on narrow tasks. What this eval shows is that AI systems are getting better at doing entire attacks end-to-end. They haven’t yet reached the “set it and forget it” level of autonomy, but they are clearly on a steep trajectory of improvement. This will lower the cost of conducting cyberattacks and multiply the number of actors that can carry them out.
Read more: How do frontier AI agents perform in multi-step cyber-attack scenarios? (AI Security Institute).China builds a dataset and AI model for electronic warfare
In scenarios such as electronic countermeasures, [systems like MERLIN] can serve as assistants in devising strategies to jam hostile signals or to counteract adversarial jamming,” the researchers write.
Who did the research: Tsinghua University, Beijing University of Posts and Telecommunications, Tianjin University, Chinese Academy of Sciences, HKUST, National University of Defense Technology (emphasis mine), Beihang University, Beijing Information Science and Technology University, and China Electronics Technology Group Corporation.
What they built: The authors built three things: a dataset, a benchmark, and a model.
- The dataset: EM-100K is a collection of 100,000 electromagnetic text-signal pairs spread across a variety of sub-tasks needed for electronic warfare, including signal classification.
- The benchmark: EM-Bench is a benchmark of 4,200 questions split across multiple choice (perception) and open-ended (reasoning) that evaluates how well AI systems can perceive and reason about EM signals across both perception and reasoning tasks, including:
- Signal characterization (modulation
Source Read original →
