AI is more likely than humans to form biases when hiring

Researchers at Princeton and the University of Chicago found that artificial intelligence systems create stronger stereotypes during hiring than people do.

The study used large language models to run a simulated recruitment game. Participants included ChatGPT, Claude, and Gemini. Each model acted as a consultant for a fictional mayor and had to fill 20 different roles. These positions ranged from doctors and lawyers to child-care aides and janitors.

Candidates came from four fictional ethnic groups: Tufa, Aima, Reku, and Weki. In every round, four applicants appeared, one from each group. The models hired one person per round and learned the outcome before moving to the next. The goal was to make as many successful hires as possible over 40 rounds. In reality, every candidate had an equal chance of succeeding regardless of their background.

The systems quickly began sorting candidates by group based on early results. If an Aima failed as a doctor, the model stopped hiring Aimas for that role. It then assigned them to jobs like janitorial work, which the AI classified as requiring less warmth and competence.

Human participants in the original psychology study scored 0.84 on a segregation scale. The models scored roughly 65% higher. OpenAI’s reasoning model o3 reached a score of 1.83, which is near the maximum possible value.

Ryan Liu, a PhD student at Princeton and coauthor of the paper, says LLMs are eager to form generalisations from limited data. That is a core part of their design. Every decision-maker faces a trade-off between sticking with what worked before and trying something new. Psychologists call this the exploration-exploitation dilemma.

Because these models are trained on math, coding, and science, they are used to settling on a hunch after just a few examples. The same instinct that helps them solve logic puzzles makes them quick to stereotype in social settings. Newer models with higher reasoning capabilities, such as o3 and DeepSeek’s R1, showed even stronger biases.

When LLMs rush to generalise in social contexts, problems tend to arise. OpenAI and Anthropic did not respond to requests for comment.

Angelina Wang, a computer scientist at Cornell University, notes this issue is especially relevant as chatbots gain improved memory and personalisation features. When a system draws on previous conversation history, it can over-index on the same kinds of behaviours it has experienced before. Wang says simply reducing memory is not a fix, because users want systems to remember what they say. The challenge is finding the right amount that is neither too much nor too little.

Telling the model to be fair did not change its behaviour much. Liu says either the systems cannot put these values into action or the process is submerged under the tendency to optimise for the goal of getting the most correct hires. Offering an additional bonus for diverse hiring made them far less biased.

The trick is to design goals that incorporate desirable social values to make large language models act in socially desirable ways.

Providing more personal information about individuals also reduced bias. In another experiment, the researchers asked the models to resettle members of different ethnic groups in cities across Canada. When the systems were told personal information relevant to adapting to a new city, such as age and education, they were less likely to segregate people by ethnicity. When given irrelevant details like hair colour or tattoo shape, the models largely fell back to sorting people by ethnicity again.

Whether AI systems will stereotype job applicants in the real world remains an open question. The models in the experiment received instant feedback on their hiring decisions. A system screening résumés in the real world does not get an immediate report card. Companies can take a long time to determine if a new hire is any good. When feedback does trickle in, a model could still read too much into those results when making future hires.

As companies increasingly deploy LLMs to screen résumés and conduct interviews, the finding that models form biases from their hiring experience is a serious implication they should grapple with. Wang says this is particularly true as these tools move into the workflow.

As LLMs learn from experience to decide who gets hired, who gets a loan, or who gets parole, the biases we should worry about may include ones no human ever taught them. Liu describes these novel biases as ever present.

What it means

For people applying for jobs, the risk is that an AI might reject a strong candidate because a previous applicant from the same group failed in a similar role. The system learns from those failures and changes its behaviour for everyone else in that group. Designers of these tools must build in specific goals that encourage diversity, rather than relying on the AI to learn fairness on its own.