By Ritendra Datta and Karthik Gopalratnam, Members of Technical Staff
OpenAI engineers built a custom advertising stack for ChatGPT because traditional event‑based ad systems don’t fit the dynamics of multi‑turn conversations. The team describes a post‑trained frontier large language model (called the orchestrator) that makes tool calls for retrieval, ranking, relevance evaluation and auctioning to select and price ads.
Why chat changes the problem
Conventional advertising reacts to discrete events (a search query or a page view). In contrast, ChatGPT users bring evolving goals across multiple turns—planning a trip, choosing software, furnishing a home or learning a hobby—so commercial intent may emerge gradually. Acting on an early message can misread intent, surface irrelevant ads or interfere with the user journey.
To address this, the team made three core design choices:
- use stateful ad sessions that remember and update information across turns and sessions;
- rely on a custom, post‑trained LLM that uses a tool‑call architecture to interpret conversation context and intent and then select, rank and deliver ads;
- run ad selection separately from ChatGPT’s answer generation in an independent service so ads cannot influence ChatGPT’s organic voice.
Overview of the ad flow
The orchestrator continuously interprets the conversation and iteratively calls ad system tools. It turns conversations into short text summaries, converts those into semantic embeddings, retrieves candidate ads from multiple indices, applies eligibility filters (budgets, targeting, policy, pacing, placement compatibility, deduplication), scores candidates for relevance and predicted engagement, and nominates contextually appropriate candidates for auction. The orchestrator does not itself set prices or unilaterally pick the delivered ad—those steps are done by the auction tool.
Retrieval intentionally produces a broad candidate pool. The orchestrator may generate multiple queries to preserve distinct strands of intent (for example, transportation versus lodging in a trip planning conversation), and the system uses approximate‑nearest‑neighbor (ANN) search over different indices optimized for various ad types.
Timing, delivery and chat specifics
Advertising runs alongside ChatGPT’s ordinary response generation but in a different service. This separation ensures ChatGPT responses are not delayed or shaped by ad logic; if the ad service is unavailable or slow, ChatGPT still responds. The system treats returning no ad as a normal, successful outcome unless there is a clear benefit for users and advertisers.
Ads are delivered server‑side as a bundle containing the creative (title, description, destination link, call to action) and a short, model‑generated neutral preamble explaining how the ad relates to the conversation. The front end will display the ad under ChatGPT’s response only if the ad bundle arrives before the response is completed; otherwise the ad is not shown for that turn. Thus winner selection and actual delivery are separate events: selection alone does not guarantee a visible impression.
Two chat‑specific failure modes are highlighted: an ad that is technically eligible may still be blocked by the orchestrator if it feels inappropriate for the conversation, and an ad request can time out if it does not complete before the model is ready to respond.
Scoring, auction and safeguards
After filtering, the orchestrator calls scoring systems that estimate semantic relevance, predicted engagement (likelihood of view/click/conversion), and historical performance. These systems narrow candidates for auction admission, where final mechanisms handle pacing, bid computation, quality‑adjusted ranking, pricing and selection. Each ad must meet high eligibility and relevance thresholds to be eligible for placement.
Monitoring, logs and self‑improvement
The ad stack uses traditional logging and data pipelines to make it easier for advertisers to integrate reporting, to enable strategy improvements, and to diagnose parts of the system independently. Logged events include outcomes such as ineligible turns, sessions with no retrieval, retrievals that failed eligibility, and ads delivered with subsequent interactions.
High‑reasoning models observe and judge system outputs and are used to improve the post‑trained LLM; smaller or more efficient models can act as workers for orchestration or retrieval once trained by the stronger models. Human feedback (clicks, hides, user surveys and advertiser feedback through Ads Manager) is explicitly incorporated and the engineering team engages directly with both users and advertisers to shape product decisions.
Early results and tooling
According to the engineers, ChatGPT Ads reached a $1 billion annualized revenue run rate in under 200 days after launch, and more than 100,000 advertisers have run ads on ChatGPT since launch. The team credits Codex as an important accelerator: they used Codex for rapid development, code guardrails and integrations that tightened feedback loops for product and quality improvements.
Future directions
OpenAI plans to experiment with more dynamic ad formats to increase user and advertiser value—for example enabling consumers to talk directly to businesses through ads—and to rely on the stack’s self‑improving architecture and continuous feedback loops to enhance relevance and personalization over time.
Conclusion
The described architecture departs from event‑driven ad models by maintaining conversation state, using a post‑trained LLM orchestrator to call modular ad tools, and keeping ad selection independent from response generation. The approach is designed to increase relevancy and preserve user trust by delivering ads only when they clearly add value.
