Less than half of the insights produced by enterprise AI agents result in actual business improvement. A new report from MIT Technology Review and Snowflake finds that while descriptive data is handled well, the prescriptive advice agents give to trigger actions is often flawed.
The study highlights a gap between the promise of agentic AI and its current reality. Automation is intended to execute workflows based on these insights, yet the quality of the output remains a significant hurdle.
Failure to generate useful analysis means organisations are missing out on their investment return. Conversely, bad advice can lead to incorrect actions, causing financial loss or reputational harm. The report examines how companies view their current agent outputs and what steps they are taking to improve them.
Key findings
Agentic analytics is only just starting to deliver. On average, 48% of insights generated by analytics agents at the surveyed companies lead to actions that improve the business. This is an advance from the 37% average recorded one year ago. However, more than half of the insights generated today still fail to help businesses.
Better quality of insights leads to clear business advantage. A small group of organisations, termed “insight leaders”, show much better performance. In this group, more than 70% of agent insights lead to actions that improve the business. These leaders rate the quality of their agents higher than the total respondent pool. The gap is particularly wide for prescriptive insights.
Insufficient context reduces insight quality. When analytics agents deliver insights of substandard quality, the main reason is a lack of business context, as cited by 55% of surveyed executives. Fragmented or inaccessible data is the other major factor, according to 45%. Fewer insight leaders report struggles with access to context.
Insight overload further inhibits agent effectiveness. At a majority (52%) of surveyed organisations, analytics agents generate too many insights to be properly evaluated and therefore acted upon. Less than one-quarter (24%) are able to evaluate all agent insights; most do so on a case-by-case basis, and that process is currently human-intensive.
Governance and semantics are areas of focus for improving insights. The surveyed companies are prioritising initiatives that enable better agent insights by strengthening AI and data governance (cited by 60%). They are also adding a semantic layer to their data architecture (45%). The experts interviewed agree that the semantic layer is crucial to ensure high-performing agents.
This content was produced by Insights, MIT Technology Review’s custom content arm, not its editorial staff. It was researched and written by humans, with any AI tools that may have been used limited to production processes under human oversight.
