OpenAI presented benchmark data for its Jalapeño custom silicon at the Hot Chips conference on Tuesday. The new chip recorded higher tokens per user and greater throughput per kilowatt than the current state-of-the-art inference processors when tested on SemiAnalysis’ InferenceX benchmark. Richard Ho, OpenAI’s head of hardware, stated the results show a very significant performance advance over existing technology. He noted the system can serve more AI work per unit of power while returning responses more quickly. The comparison was against an Nvidia Blackwell system, though Ho estimated Jalapeño would deploy in very small volumes by the end of 2026 with larger scale arriving in 2027.
The hardware was developed by OpenAI in close collaboration with Broadcom, with the company’s own models assisting the design process. This full-stack approach allows Jalapeño to address specific phases in the inference process that often cause friction. The chip minimises delays during the prefill and communication phases by reducing data movement and keeping model state local. It activates the right combination of compute, memory, and networking for each inference phase. This design aims to lower latency while serving many customers efficiently.
- Deployment begins in very small volumes at the end of 2026
- Larger scale deployment is planned for 2027
- Jalapeño is designed to minimise data movement during inference
