A UK government watchdog is considering an investigation into the Home Office’s use of generative AI tools to process asylum applications. This potential probe follows public questioning from lawmakers and researchers regarding the accuracy of the software and the government’s secrecy around its deployment.
The watchdog’s stance
The Independent Chief Inspector of Borders and Immigration has asked digital rights nonprofit Open Rights Group to share more information on the tools as it plans its inspection priorities for next year. Campaigner Sara Alsherif told WIRED that the request comes from the inspectorate’s own inquiry process. ORG’s findings stem from a six-month investigation involving 11 parliamentary questions, two ministerial letters, and 10 public records requests aimed at extracting details the Home Office has withheld.
The ICIBI cannot confirm which topics are under active consideration before publishing annual priorities. It stated it is aware of concerns about the use of AI tools and pointed to last year’s findings that one tool’s effectiveness and reliability were varied.
“Asylum decisions are made by trained caseworkers, with AI tools used only to assist in navigating interview records and policy information. They cannot be used in isolation and do not make decisions on claims,” a Home Office spokesperson told WIRED in an email. “We regularly monitor the performance of these tools alongside our established quality-assurance processes, and have already committed to publishing updated evaluations on their roll out.”
How the tools work
The Home Office uses the Asylum Case Summarisation (ACS) and Asylum Policy Search (APS) tools. These systems are built on OpenAI’s 2023 GPT-4 large language model and Microsoft Foundry. They summarise transcripts of interviews with asylum applicants and information about their countries of origin, according to documents obtained by ORG.
The immigration and law enforcement authority started piloting these tools in late 2024. The Labour government aimed to clear a backlog of around 100,000 asylum cases while managing tens of thousands of new applications received each year. The department reported significant progress, halving the number of people waiting on an initial decision in the year through June. The number of appeals against decisions has increased.
Researchers are concerned about the accuracy of the AI tools, especially when deciding whether to send someone back to a country where they fear persecution. ORG found that the percentage of summaries generated by the ACS deemed not fit for use is significantly higher than the published 9 percent error rate calculated in the pilot evaluation. These summaries often contain factual errors or misrepresentations. There is no quality control for the summaries produced by the APS, it also found.
Asylum decision-makers use the tools to save time when reviewing cases. When used together, the pilot showed the tools could save around an hour per case. Caseworkers use the ACS to speed up their review of transcripts where asylum seekers explain how they were persecuted and why they are afraid to return. They use the APS as a search assistant to find and summarise country policy information, which informs them whether the applicant faces a genuine risk of harm in their home country.
In July, a UK judge accused the Home Office of citing an “AI hallucinated” document to refuse a Moroccan woman and her child’s asylum claim on the basis that her home country would be safe for her to return to.
Human oversight
“We’re dealing with people’s lives. We’re talking about people that have faced some of the worst circumstances, and we’re meant to be making decisions based on facts and on understanding harm,” Labour backbench MP Bell Ribeiro-Addy told WIRED. “I just don’t think it’s appropriate that machines are doing that.”
A Home Office spokesperson said, “Use of Artificial Intelligence is just one of the tools at our disposal to protect our borders, and does not replace or automate the role of human decision-makers.”
Ribeiro-Addy, Alsherif, and the Public Law Project also worry that well-documented “cultural bias” in GPT models could lead to language barriers and the loss of cultural nuances in the summaries. This could result in discriminatory decisions. An OpenAI spokesperson highlighted to WIRED the company’s ongoing work to improve its models’ accuracy and pointed out that the company will be retiring GPT-4 in January.
Transparency gaps
In the absence of regulation, the UK’s “AI Playbook” mandates that departments must be transparent about the algorithmic tools they use in decision-making processes. They must make information on them available to the public, including listing those tools on a public repository. The Home Office’s case summarization and policy search tools are not on that list.
Asylum seekers are also not told when these AI tools are used in their case. “Migrant organizations, legal practitioners, not many of them know about the actual use of this tool,” says Alsherif, who leads ORG’s migrant digital justice program. “So we need to let them know about this, let them know how it works, so they can challenge it.” Not being able to contest an incorrect AI summary of their personal information may breach UK data rights laws, Cloisters lawyers wrote in an independent legal opinion commissioned by ORG. The Home Office states that parts of the decision-making process may involve automation in its privacy notice.
Both groups have sent their findings to UK data regulator, the Information Commission, which is currently investigating the Home Office’s refusal to tell ORG how staff use and were trained to use two AI tools.
Public Law Project senior research fellow Caroline Selman noted that the ACS and APS tools are just two of the 11 high-risk AI systems her team has found being used across government but absent from the public repository. She is concerned there may be more deployed across government that even researchers, let alone the individuals they are being used on, are unaware of.
“Other tools are being used but not being publicized at all,” says Selman, “and you can’t even begin to ask questions about things if you don’t know that they’re being used.”
