Google announced that agentic video understanding is now available in its latest Gemini models: Gemini 3.7 Flash, 3.6 Flash and 3.5 Flash‑Lite. The new capability is designed to increase accuracy in video analysis while substantially reducing token usage and costs.
What agentic video understanding does
Agentic video understanding departs from the previous static processing approach. Instead of ingesting video at a fixed frame rate (default 1 FPS, adjustable via API), the system combines the model’s reasoning with native video tools to actively search, scan and inspect relevant segments across frames, audio and transcripts. Gemini runs an agentic loop that invokes internal tools to load only the needed parts of the video file, which reduces development overhead.
Benchmarks and efficiency
According to internal benchmarks, Gemini models with agentic video understanding can reduce analysis costs by up to 66% and token consumption by up to 88%, while improving accuracy by up to 7% on standard video analysis tasks. These efficiency gains are especially notable for long‑form video (from 10‑minute how‑to guides to 90‑minute lectures and multi‑hour recordings), where static processing forces tradeoffs between high token costs and dropping critical details.
All three supported models benefit from agentic processing, but Gemini 3.7 Flash is highlighted as delivering the best overall quality and the most favorable quality‑to‑cost balance, placing it on the accuracy‑to‑cost pareto frontier among tested models.
Key capabilities and use cases
- Sub‑second moment retrieval: Identify split‑second state changes and tight edit boundaries that 1 FPS sampling can miss, enabling precise automated editing.
- Needle‑in‑a‑haystack search across long video: Answer complex queries across multi‑hour recordings without consuming millions of tokens.
- Anomaly detection: Resample interesting windows at higher FPS to inspect rapid motion or subtle visual artifacts.
- Counting actions and objects: Track repeated physical movements and distinct objects over time with improved accuracy.
How it works in practice
With agentic video understanding, the model decides what to watch, at what speed, and through which modality (frames, audio, or transcript), fetching only the signals required for the task. For example, Gemini 3.7 Flash can accurately count a fast‑paced action by scanning and rewatching segments at different FPS as needed.
Availability and pricing
Agentic video understanding is available today for video uploads and YouTube videos via the Gemini API in Google AI Studio and the Gemini Enterprise Agent Platform. It uses standard Gemini API token pricing with no additional feature fee. To enable the capability, set processing to "agentic" in the API configuration. Google also provides a developer guide with implementation details.
Rollout beyond APIs
Google said it will bring the efficiency and quality improvements to a broader user base: the feature will roll out to all users in the Gemini app across Flash and Flash‑Lite models soon. In the coming months, agentic video understanding will also power YouTube’s "Ask YouTube" feature on the video watch page to provide higher‑quality answers grounded in visuals.
Acknowledgements
Contributors named for this work include Sergi Caelles, Filip Pavetić, Ahmet Iscen, Suhas Yogin and the Agentic Vision team.
Conclusion
Agentic video understanding enables Gemini models to act in a goal‑directed way when processing video, aiming to deliver faster, more accurate results while using far fewer tokens and lowering costs — advantages that are particularly meaningful for long and detail‑rich video content.
