Humans still outperform artificial intelligence on specific logic puzzles, even as models rapidly close the gap on others. While AI systems solved 18% of the New York Times Connections puzzles in late 2024, some now handle them near perfectly. Yet subtle changes in classic riddles and visual tasks continue to trip them up.
This gap reveals where machine cognition diverges from human thought. You can test your own wits against these specific challenges. Some are difficult for you; others are so simple they cast doubt on whether AI is truly intelligent.
Spatial Reasoning
Humans hold a distinct advantage in spatial reasoning. IQ tests often include mental rotation problems, which ask you to identify if different images represent the same object viewed from various angles. Current language models struggle with this.
Despite claims that world models help AI understand physical environments, these systems cannot manipulate 3D objects like architects or mechanical engineers do. They fail abysmally at tasks requiring genuine spatial manipulation.
Mental Rotation
Instructions: Choose the answer that shows the object in the prompt, but from a different angle. In each case, there’s only one correct answer!
Memory & Adaptability
Frontier large language models possess vast memories. Their training exposed them to a massive volume of facts, allowing them to recite information faithfully. This strength is an asset for trivia but a liability for puzzles.
When a puzzle resembles one the model saw during training, the system often ignores key differences and responds with memorised answers. A 2024 study by researchers from Google and the University of Illinois Urbana-Champaign demonstrated this on slight variations of the Knights and Knaves puzzle.
In these problems, some characters always tell the truth while others always lie. You must determine who is who. The same principle applies to SimpleBench. These questions resemble complicated problems models likely encountered during training. Humans spot the trick; even top-tier models trip.
Knights and Knaves
Instructions: The only thing you need to know to solve these puzzles is that knights always tell the truth and knaves always lie. Determine who’s what on the basis of what each character says.
SimpleBench
Instructions: Read these SimpleBench problems carefully, and you should be able to figure out the answers in no time.
Abstract & Visual Reasoning
AI struggles with visual problems in two dimensions as well as three. This is a major factor in performance on ARC-AGI, the most famous puzzle-based benchmark. These tasks require inferring abstract, general rules from a set of examples.
Models perform better on ARC puzzles when they receive each grid as a string of numbers encoding cell colour rather than as an image. Research suggests that even when models answer correctly, they often use byzantine and non-generalisable rules, whereas humans draw on simple visual concepts.
Despite these disadvantages, models have improved at ARC-AGI over the past year. However, some puzzles, including the one printed here, still stump them.
ARC-AGI
Instructions: Study the three pairs of grids shown below to figure out the rule that dictates how the ones on the left transform into the ones on the right. Then get out your markers or colored pencils and fill in the fourth grid using that rule. (The solution is the same no matter which way the grids are oriented.)
Intuition
Humans fall into traps that AI does not share. Psychologists have designed problem suites that invert the SimpleBench phenomenon. For these questions, humans often give knee-jerk answers, whereas models respond deliberatively.
Some problems exploit errors in how we intuitively do math. Others are phrased to suggest obvious answers that fall apart if the question is read carefully.
Lightning Round
Instructions: Answer the questions below as quickly as you can.
Increasing Complexity
Whether an LLM can complete a puzzle depends on scale. One study from Apple researchers found that LLMs can ace simple versions of the Tower of Hanoi problem and river-crossing puzzles. These tasks involve moving a stack of disks one at a time without putting a larger disk atop a smaller one, or a group of people traversing a river according to certain rules.
The systems succeed only up to a point. As the number of disks or people hits six and higher, the models began to falter. Another study by researchers at the University of Washington, Stanford University, and the Allen Institute for AI observed similar struggles with logic grid puzzles.
These puzzles require deducing the attributes of a set of individuals from a list of clues. The Apple paper went viral, but commentators questioned whether the results reveal a unique limitation of LLM reasoning or just that it is normal to make errors as complexity piles up.
The River
Instructions: Using the scenario provided, plan the trips necessary to get everyone across the river.
Logic Grid
Instructions: Using the list of clues, determine who lives in each house and what style of music each person enjoys. There is only one possible solution. You may find it helpful to fill out the grid below to keep track of your deductions.
What it means
For creators and developers, these results highlight a specific weakness in current AI. Systems rely heavily on pattern matching and memorised training data. They struggle when a problem requires genuine spatial manipulation, simple visual intuition, or careful reading of subtle constraints.
This suggests that simply scaling up data volume will not solve these issues. Models need better ways to represent physical space and handle logical deductions that diverge from their training distribution.
