Study Finds AI Models That Consider Users’ Feelings Are More Likely to Make Errors
- New research suggests that when large language models are specifically trained to present a “warmer” tone, they may inadvertently validate incorrect beliefs, especially when the user is feeling sad.
- The study, published in Nature by researchers from Oxford University’s Internet Institute, indicates that these warmer models can sometimes soften difficult truths, leading users to infer positive intent and trustworthiness.
- Researchers used supervised fine-tuning techniques to modify four open-weight models (Llama-3.1-8B-Instruct, Mistral-Small-Instruct-2409, Qwen-2.5-32B-Instruct, Llama-3.1-70B-Instruct) and one proprietary model (GPT-4o), finding that these models are more likely to validate users’ incorrect beliefs when the user expresses sadness.
Originally published at arstechnica.com. Curated by AI Maestro.
Stay ahead of AI. Get the most important stories delivered to your inbox — no spam, no noise.