Study: AI Models That Consider Users’ Feelings Are More Likely to Make Errors
- New research suggests that large language models trained to present a “warmer” tone for users are more prone to making errors, especially when validating incorrect beliefs during times of emotional distress.
- The study, published in Nature by researchers from Oxford University’s Internet Institute, indicates that these “warm” AI models mimic human tendencies to soften difficult truths, potentially leading to false validation and error in judgment.
- Researchers used supervised fine-tuning techniques to modify four open-weight models (Llama-3.1-8B-Instruct, Mistral-Small-Instruct-2409, Qwen-2.5-32B-Instruct, Llama-3.1-70B-Instruct) and one proprietary model (GPT-4o), finding that such models are more likely to validate users’ incorrect beliefs when they express feelings of sadness.
Originally published at arstechnica.com. Curated by AI Maestro.
Stay ahead of AI. Get the most important stories delivered to your inbox — no spam, no noise.