Google has made its Gemini Flash models capable of deciding which parts of a video to watch, allowing them to cut video token usage by up to 88% and reduce costs by 66%.
Previously, the system ingested a full video timeline at a fixed one frame per second. This approach forced developers to choose between paying to process the entire duration or manually pre-chunking files, a step that often resulted in missing key details.
The new agentic video understanding feature changes this workflow. Instead of a single pass, the model navigates the timeline autonomously. It determines the frame rate, selects specific segments, and chooses the appropriate modality—video, audio, or transcript—based on the user’s prompt. Google reports that this method improves accuracy by up to 7% on standard benchmarks compared to the old static processing.
Deployment and availability
Access is currently restricted to hosted API features. There are no open weights and no option for self-hosting. The feature ships through the Gemini API in Google AI Studio and the Gemini Enterprise Agent Platform.
Developers can upload local files or provide public YouTube URLs. Billing follows standard Gemini API token pricing with no additional feature fee attached.
How the processing differs
Static processing remains the default for most Gemini models. That method extracts frames at 1 FPS in a single pass, processes audio at 1 Kbps on a single channel, and inserts timestamps every second.
Agentic processing replaces this linear approach with a loop. The model pairs its own reasoning with native video tools to search, scan, and inspect target segments. It loads only the specific data required by the prompt. While developers could previously assemble this logic manually, the change is that Gemini now runs the loop internally, removing the development overhead.
Evaluations across Google’s test suite show that Gemini 3.7 Flash with agentic understanding lands on the accuracy-to-cost Pareto frontier for video analysis among the models tested. The efficiency gains concentrate on long-form content, ranging from 10-minute how-to guides to multi-hour recordings.
API responses and billing
Agentic processing adds two step types to the response steps array. A processing_call appears when the model requests a segment or transcript. A matching processing_result appears when that load completes. These steps interleave with thought steps and precede model_output.
This structure allows developers to drive a live progress trace in their user interface. The presence of these steps also serves as verification that agentic mode actually ran.
Token accounting splits accordingly. Navigation reasoning bills as thought tokens (total_thought_tokens). Frames, audio, and transcripts loaded on demand bill as tool-use tokens (total_tool_use_tokens).
To enable the feature, developers add one field to the video part of the request:
interaction = client.interactions.create(
model="gemini-3.7-flash",
input=[
{
"type": "video",
"uri": "https://youtu.be/7Z5Vy9JBANs",
"processing": "agentic"
},
{
"type": "text",
"text": "What are the 3 most important announcements in this keynote?",
},
],
)Users can mix modes per video inside a single request. Agentic mode handles the long lecture while static mode processes short clips.
Supported models and limitations
The feature is supported on Gemini 3.8, 3.7, 3.6 Flash and 3.5 Flash-Lite. Static processing remains better for clips under five minutes and for frame-by-frame precision work.
What it means
The shift to agentic processing means the cost of handling long video content drops sharply. The model no longer pays to process empty air or irrelevant sections of a timeline. This allows complex queries on long-form content without the previous trade-off between cost and detail.
