Hundreds of enterprise leaders and technical experts filled the main ballroom of Hotel Nia in Menlo Park this week for VB Transform 2026, a conference focused on using generative AI agents to drive business outcomes. Rachad Alao, vice president of product engineering at Cohere, spoke with Matt Marshall, CEO and editor‑in‑chief of VentureBeat, about building agentic systems without relinquishing control of sensitive data, infrastructure, or vendor choice.
Alao, who previously led responsible AI and trust and safety engineering teams at Google and Meta, argued that AI sovereignty is more than downloading an open model or running an application behind a corporate firewall. He pointed to organizations operating mission‑critical systems—banks, hospitals and governments—where tight control over data location and AI behavior is essential.
Sovereignty, he said, extends from GPUs and private‑cloud infrastructure through governance systems that route requests between models, to connectors, search tools and agent frameworks acting on enterprise data. “You want to have control on the entire stack,” Alao summarized.
Token consumption and agent workloads
Marshall questioned one economic argument for smaller, locally deployed models: inference prices are falling rapidly, which might weaken the case for optimizing every token. Alao replied that total consumption is rising even faster as enterprises move from simple chatbots to complex, multi‑step agents that reason through problems, call tools, search internal systems and take multiple steps before returning answers.
“Your token utilization is going exponentially up, because you’re dealing with more and more complex agentic use cases,” he said, noting these workflows require significant processing, reasoning and tool interaction.
Alao contrasted providers that bill customers by token consumption with Cohere’s approach. He argued that token‑based charging incentivizes maximizing token use, whereas Cohere does not sell its models and platform that way. Instead, Cohere aims to help enterprises solve hard problems privately and securely while reducing unnecessary model usage. His prescription: “Use the right model for the task at hand.”
Rather than routing every request to the largest frontier model, enterprises should choose models based on the intelligence required and the sensitivity or regulatory burden of the task. Alao cited an unnamed Canadian bank that uses Cohere’s on‑premises models for highly regulated workloads and routes less sensitive but more intelligence‑demanding tasks through Cohere’s North platform to larger frontier models. “So model routing can become super useful,” he said.
Smaller models for most enterprise work
An audience member asked how Cohere’s open‑source North Mini Code, released last month, can compete with proprietary coding models. Alao conceded that larger frontier models may perform better on the hardest problems, but said that advantage does not justify indiscriminate use.
“For 80% of the use cases that they needed, this was a lot more effective, a lot cheaper,” he said of developers adopting the model. North Mini Code runs on a single Nvidia H100 GPU and targets agentic software engineering tasks such as terminal work, code review and tool use.
Cohere also released Command A+, a 218‑billion‑parameter mixture‑of‑experts model with only 25 billion parameters active during each generation step. Its compressed four‑bit variant reduces the hardware required for private deployment, and the Apache 2.0 license gives enterprises broad freedom to operate and modify the model.
Search as part of the agent
On the topic of embeddings and enterprise search, Alao said the field is moving beyond retrieving text and pasting it into a model’s context window. “Today, the state of the art is around multimodal search,” he said. Search across documents, images and other information types is becoming an integral component of agentic workflows, with the model deciding when and how to use retrieval like any other tool.
Asked what would persuade enterprises to move beyond bundled AI services from existing cloud providers, Alao again returned to data control and portability. Cohere’s governance layer, he said, lets customers route traffic to appropriate models, addressing the vendor lock‑in concerns that worry many of their customers.
