At the Hot Chips conference on Tuesday, OpenAI provided a deeper look at its Jalapeño inference chip and released the first benchmark results for the system. In tests run on the SemiAnalysis InferenceX benchmark, OpenAI says Jalapeño delivered more tokens per user and higher throughput per kilowatt than currently available state-of-the-art inference processors.
Company statements and comparison
Richard Ho, OpenAI’s head of hardware, said on a press call: “The bottom line is that the results show a very, very significant performance advance over state of the art. Jalapeño can serve more AI work per unit of power, while also returning responses more quickly. It’s very efficient to serve a lot of customers, but it can also be very low latency.”
OpenAI noted that the benchmark comparison was made against an Nvidia Blackwell system. The company also acknowledged that competing hardware may advance further by the time Jalapeño reaches full deployment.
Deployment timeline
Ho estimated a limited deployment of Jalapeño at the end of 2026, describing it as "in very small volumes," with more substantial deployment planned for 2027.
Development partners and platform approach
Jalapeño was first announced in October of the previous year. The chip was developed in close collaboration with Broadcom, and OpenAI’s own models were used to assist the development process. OpenAI intends for Jalapeño to serve as a multigenerational platform, enabling coordinated development of AI products, models, chips, and memory.
Because of this full-stack approach, OpenAI said it could target specific bottlenecks that often appear during inference. In particular, Jalapeño’s design aims to reduce delays during the prefill and communication phases of processing, which OpenAI identifies as common sources of latency.
In a blog post presenting the results, the company wrote: “We designed Jalapeño to minimize data movement and communication delays. This means that model state, including the KV cache used while generating a response, can be explicitly placed and kept local while the system activates the right combination of compute, memory, and networking for each inference phase.”
Significance
If the benchmark results hold in broader deployments, Jalapeño could offer improvements in latency and energy efficiency important for large-scale, real-time AI services. OpenAI also cautioned that the eventual market impact will depend on competitor progress and on how the platform rolls out in 2026–2027.
Summary
OpenAI’s Jalapeño shows promising gains in performance and power efficiency in initial benchmarks, with limited deployment expected at the end of 2026 and wider rollout in 2027.
