Cloudflare has introduced a new option that enables its customers to charge AI models and automated agents that scrape content from their websites. The company’s architecture functions like a bot paywall: content owners can set a minimum price, AI crawlers can submit a maximum bid, and the platform matches requests according to the dynamics of a two-sided marketplace.
Who is affected?
The change primarily affects online retailers, travel companies, and publishers — organizations that have long negotiated with search engines and platforms over how to be compensated for web content. Large language models (LLMs) and other automated scrapers now present a similar challenge by extracting and reusing content without necessarily returning direct revenue to the source sites.
Key figures and claims
Cloudflare says that automated agents account for an increasing share of web traffic: Stephanie Cohen, Cloudflare’s chief strategy officer, told Semafor that more than half of its web requests now come from agents. This shift is a central reason the company has developed tools specifically aimed at monetizing AI-initiated requests.
The company’s rationale and market mechanics
Cloudflare frames the development around the idea that the internet is moving toward a situation where "every interaction, every transaction, is a chance for commerce," as Cohen put it. Under the new architecture, content providers set a floor price for access to their material, while AI systems can indicate a maximum bid; the platform then uses those parameters to determine which requests are fulfilled in a market-based pairing.
Cohen also argued that subscription models built by media companies are not necessarily doomed by this change. "I don't think everyone wants to read the average," she said, suggesting that outlets offering distinctive, in-depth, or voice-driven content may retain value for paying audiences.
Why it matters
The system gives web owners a way to monetize AI-driven scraping and reconnect some of the value of their content to its origin. At the same time, the rollout raises technical, legal, and market questions: how access will be measured, what effects the mechanism will have on search and AI functionality, and how payment and licensing practices will evolve in practice.
Cloudflare’s move signals that major infrastructure providers are actively seeking business models to address the economic implications of AI agents’ growing presence online.
