OpenAI has released the first detailed performance results for Jalapeño, its custom AI inference chip, and the early numbers suggest that the company may have found a way to make AI responses both faster and more power-efficient.
The chip, developed with Broadcom, is designed specifically for large language model inference — the stage where a trained AI model generates responses for users. In early testing using the SemiAnalysis InferenceX benchmark, OpenAI says Jalapeño delivered more AI work per unit of power while also achieving lower end-to-end latency than the Nvidia systems used for comparison.
The results are important because OpenAI is one of the world’s largest consumers of AI computing infrastructure. Even relatively small efficiency improvements can become significant when multiplied across massive numbers of AI requests.
What Is OpenAI’s Jalapeño Chip?
Jalapeño is OpenAI’s first custom AI inference processor, developed in partnership with Broadcom.
OpenAI and Broadcom unveiled the chip in June 2026, describing it as an accelerator built specifically around the requirements of modern LLM inference. The companies said its architecture was designed around AI-model kernels, memory movement, networking and serving patterns rather than trying to build a general-purpose accelerator.
That distinction is important.
Jalapeño is not intended to replace every type of AI accelerator.
Its primary job is to run already-trained AI models efficiently.
That means the chip is focused on the computational workload behind services such as AI assistants and agentic applications.
What Did the First Tests Show?
OpenAI says Jalapeño was tested using the SemiAnalysis InferenceX benchmark.
The early results showed two major advantages:
- Higher AI throughput per unit of power
- Lower end-to-end response latency
Across the tested workloads, OpenAI reports roughly 1.5x to 1.9x more AI work per watt and 1.7x to 3.6x lower end-to-end latency compared with the Nvidia systems used in the benchmark comparisons.
Those numbers are significant because inference hardware normally involves a trade-off.
A system optimized for maximum throughput may not necessarily provide the lowest response latency.
Jalapeño was designed to address both requirements simultaneously.
What Does “Inference” Mean?
AI inference is the process of using a trained model to produce an output.
For a chatbot, inference happens when you send a prompt and the model generates the response.
Training is different.
During training, enormous amounts of data are processed to adjust the model’s parameters.
After the model has been trained, inference hardware is used to serve that model to users.
That distinction explains why Jalapeño’s design is so specialized.
OpenAI is not trying to create a universal replacement for all AI hardware.
It is optimizing a critical part of its own AI infrastructure: serving models at enormous scale.
Jalapeño vs Nvidia Hardware
The first benchmark comparisons put Jalapeño against high-end Nvidia systems, including GB200 and GB300 configurations.
OpenAI reports that Jalapeño delivered higher performance per watt and lower latency across several tested models.
The tests included workloads involving models such as:
- GPT-OSS 120B
- DeepSeek R1
- Kimi K2.5 1T
The Kimi K2.5 test is particularly interesting because the model contains around one trillion parameters and represents the type of large model that places significant demands on inference infrastructure.
However, these results should be interpreted carefully.
They are benchmark results for specific workloads, not proof that Jalapeño is universally faster than every Nvidia accelerator in every AI application.
Hardware performance depends heavily on the model, software stack, batch size, memory configuration, networking and workload characteristics.
How Much Faster Is Jalapeño?
The headline numbers vary depending on the workload and metric.
OpenAI reports:
1.5x–1.9x more AI work per watt
and
1.7x–3.6x lower end-to-end latency
against the comparison systems in the reported tests.
In other words, Jalapeño’s advantage is not simply about generating more tokens.
It is also about how efficiently the system uses electricity while serving those tokens.
That distinction becomes increasingly important as AI data centers consume enormous amounts of power.
Why Performance Per Watt Matters
AI inference is becoming an energy-intensive business.
Every time someone sends a request to an AI model, computing resources have to process that request.
At small scale, a few percentage points of efficiency may not matter much.
At OpenAI’s scale, however, the economics change dramatically.
Suppose a hardware improvement allows the same amount of AI work to be completed using significantly less power.
Across thousands or millions of accelerators, that can potentially reduce:
- Electricity consumption
- Cooling requirements
- Data-center operating costs
- Infrastructure requirements
The savings can become especially important as AI usage continues to grow.
This is one reason major AI companies are increasingly designing their own chips.
Why Lower Latency Matters for AI Agents
Latency is particularly important for AI agents.
A conventional chatbot can often tolerate a short wait for a response.
An AI agent may need to perform multiple steps.
For example:
User request → model reasoning → tool call → result → model reasoning → another tool call → final answer
If every step takes longer, the total time required to complete a task increases.
Reducing inference latency can therefore make agentic systems feel much more responsive.
OpenAI has specifically designed Jalapeño around the needs of modern LLM serving, including interactive workloads.
That makes the latency results arguably just as important as the raw throughput numbers.
Jalapeño Was Built Around the Full AI Serving Stack
One of the more important details about Jalapeño is that OpenAI did not design it in isolation.
The company says the chip was optimized around:
- Model kernels
- Memory movement
- Networking
- Scheduling
- Serving patterns
- Latency requirements
OpenAI’s approach is essentially to optimize the entire path between an AI model and the user rather than focusing exclusively on the processor itself.
This is an increasingly important strategy in AI infrastructure.
A faster chip does not automatically produce a faster AI service.
Memory, networking, software and model architecture can all become bottlenecks.
Why OpenAI Wants Its Own AI Chip
OpenAI currently relies heavily on external hardware providers for its enormous computing needs.
Nvidia remains a major supplier of AI accelerators, and OpenAI has continued to work with Nvidia even while developing its own silicon.
So why build a custom processor?
There are several reasons.
Lower Inference Costs
If OpenAI can perform more AI work using less electricity and infrastructure, its cost per request could potentially decline.
Greater Hardware Control
Custom silicon gives OpenAI more control over how its models interact with the underlying hardware.
Better Model Optimization
OpenAI can design hardware around the characteristics of its own models and future inference workloads.
Reduced Dependence
Building internal hardware can reduce reliance on a small number of external accelerator suppliers.
Scaling AI Services
As AI usage grows, hardware efficiency becomes increasingly important.
The strategy therefore isn’t necessarily about replacing Nvidia.
It is about giving OpenAI another layer of control over the infrastructure underneath its products.
Jalapeño Is Not a Training Chip
One of the most important limitations is that Jalapeño is focused on inference.
It is not designed to replace the hardware OpenAI uses to train frontier models.
Training requires different capabilities and can involve enormous computational workloads with different memory, precision and flexibility requirements.
That means OpenAI will continue to need other accelerators for training even if Jalapeño becomes a major part of its inference infrastructure.
This is why calling Jalapeño an “Nvidia killer” would be misleading.
The chip is targeting a particular part of the AI computing stack.
What Hardware Does Jalapeño Use?
Technical reporting on the chip indicates that Jalapeño uses six HBM4 memory stacks, providing approximately 216 GiB of high-bandwidth memory and around 15.4 TB/s of memory bandwidth. The chip is designed around a roughly 700-watt power envelope.
Those specifications highlight the problem Jalapeño is trying to solve.
Modern LLM inference is often limited not only by raw compute but also by the speed at which model data can move between memory and processing units.
High-bandwidth memory is therefore a critical part of the system.
What Models Were Tested?
OpenAI’s early benchmark results included several demanding AI workloads.
The reported tests included:
GPT-OSS 120B
A large open-weight language model from OpenAI.
DeepSeek R1
A reasoning model known for demanding inference workloads.
Kimi K2.5 1T
A roughly one-trillion-parameter model that provides a useful test for large-scale inference infrastructure.
Testing across different models is important because a chip can perform differently depending on the architecture and workload.
A strong result on one model does not automatically translate into the same advantage everywhere.
Is Jalapeño Available to Other Companies?
No.
Jalapeño is being developed as part of OpenAI’s own infrastructure strategy rather than as a commercial chip that customers can simply purchase.
That means the immediate value of the chip is primarily internal.
OpenAI can use it to improve the economics and responsiveness of its own AI services.
This is fundamentally different from Nvidia’s business model, where the company’s accelerators are sold to a wide range of cloud providers and AI companies.
When Will OpenAI Deploy Jalapeño?
OpenAI previously said the first-generation chip was intended for deployment at large scale.
Current reporting indicates that limited deployment is expected by the end of 2026, with broader scaling planned for 2027.
That timeline matters because laboratory benchmarks are only the first step.
The real test will be what happens when Jalapeño operates inside production data centers handling large numbers of real-world AI requests.
Production environments introduce additional challenges involving:
- Networking
- Reliability
- Cooling
- Software compatibility
- Scheduling
- Supply chains
- Maintenance
- Workload variability
A chip that performs well in a controlled benchmark still has to prove itself at data-center scale.
What Does This Mean for Nvidia?
Jalapeño is a meaningful development for Nvidia, but it should not be interpreted as an immediate threat to Nvidia’s entire AI business.
Nvidia’s advantage comes from much more than individual GPU performance.
Its ecosystem includes:
- GPUs
- Networking
- CUDA
- Developer tools
- Software libraries
- Cloud partnerships
- Large installed base
OpenAI itself continues to use Nvidia hardware.
The more accurate interpretation is that AI customers are becoming increasingly interested in custom silicon for specialized workloads.
Google has its TPUs.
Amazon has Trainium and Inferentia.
Microsoft has Maia.
Meta has developed its own AI accelerators.
OpenAI is now joining that broader movement.
The Bigger Shift: AI Companies Are Becoming Chip Designers
The rise of custom AI chips reflects a deeper change in the industry.
For years, AI companies could largely purchase computing power from hardware vendors.
As AI workloads have become more specialized and expensive, that approach is becoming less attractive for some of the largest companies.
Designing custom silicon allows companies to optimize around their own workloads.
That can potentially produce better economics than using a general-purpose accelerator for every task.
The trade-off is complexity.
Chip design requires enormous investment, specialized engineering talent and long development cycles.
OpenAI’s partnership with Broadcom helps address some of that complexity.
What Are the Limitations of the Early Results?
The Jalapeño results are promising, but several caveats matter.
The Benchmarks Are Early
These are initial performance results rather than years of production data.
OpenAI Has a Stake in the Outcome
OpenAI designed the chip and is reporting results from its own infrastructure strategy.
Independent benchmarking is therefore important.
SemiAnalysis’ InferenceX testing provides an external benchmark component, but the broader claims should still be evaluated as more production data becomes available.
Inference Is Only One Part of AI Computing
Jalapeño does not solve the training problem.
Software Matters
The performance advantage depends on the software and model stack being optimized effectively.
Production Scale Is Different
Real-world deployment can expose bottlenecks that laboratory benchmarks do not reveal.
Affitronix Analysis
Jalapeño’s first benchmark results are interesting for a reason that goes beyond the headline speed numbers.
OpenAI is trying to control more of the AI compute stack.
The company already develops models and operates consumer and enterprise AI products.
Now it is increasingly designing the hardware underneath those systems.
That vertical integration could become strategically important.
If OpenAI can build hardware specifically around how its models perform inference, it does not have to wait for every optimization to come from a general-purpose chip vendor.
It can coordinate the model, compiler, runtime, memory system, networking and processor around the same workloads.
That could make the biggest difference not in a benchmark chart but in cost per AI interaction at enormous scale.
The performance-per-watt results are therefore arguably more important than the raw speed results.
If Jalapeño can consistently produce more AI work for each unit of electricity, OpenAI could potentially serve more users without increasing infrastructure costs at the same rate.
But the company still has a long way to go.
Jalapeño is an inference processor, not a universal AI accelerator.
Nvidia remains deeply embedded in OpenAI’s infrastructure, and training frontier models requires different hardware capabilities.
The more realistic future is therefore not:
OpenAI replaces Nvidia.
It is:
OpenAI uses Nvidia and other accelerators where flexibility is needed, while custom Jalapeño processors handle inference workloads where specialization can provide an economic advantage.
That is a much more credible and potentially more important strategy.
What Happens Next?
The next milestone will be production deployment.
Watch for:
- Real-world Jalapeño deployment numbers.
- Production inference costs.
- Energy consumption at scale.
- Latency under real user workloads.
- Support for future OpenAI models.
- The next generations of OpenAI’s custom processors.
- How much of OpenAI’s inference workload eventually moves to its own silicon.
The most important evidence will come when Jalapeño is operating continuously inside large-scale AI infrastructure.
At that point, the question will shift from:
“Is the chip fast?”
to:
“Can OpenAI make AI meaningfully cheaper and more responsive with its own hardware?”
Final Takeaway
OpenAI’s Jalapeño has delivered strong early inference results, with the company reporting 1.5x to 1.9x more AI work per watt and 1.7x to 3.6x lower end-to-end latency than the comparison Nvidia systems in the reported tests.
The chip is designed specifically for LLM inference, not AI training, and is being developed with Broadcom as part of OpenAI’s broader effort to control more of its computing infrastructure.
The real significance is not that OpenAI has suddenly replaced Nvidia.
It is that one of the world’s largest AI companies is now building specialized hardware around its own models and workloads.
If those efficiency gains survive production deployment, Jalapeño could help OpenAI reduce the cost and latency of serving AI at massive scale.
FAQ
What is OpenAI’s Jalapeño chip?
Jalapeño is OpenAI’s first custom AI inference processor, developed with Broadcom specifically for large language model inference.
How fast is OpenAI’s Jalapeño chip?
OpenAI reports that Jalapeño achieved about 1.5x to 1.9x more AI work per watt and 1.7x to 3.6x lower end-to-end latency than the Nvidia comparison systems in its early benchmark testing.
Is Jalapeño designed for AI training?
No. Jalapeño is primarily an inference chip designed to run trained AI models efficiently. OpenAI still requires other hardware for training frontier models.
What AI models were tested on Jalapeño?
The reported benchmark testing included GPT-OSS 120B, DeepSeek R1 and Kimi K2.5 1T, among other workloads.
Is OpenAI selling Jalapeño chips?
No. Jalapeño is being developed for OpenAI’s own infrastructure rather than as a commercially available accelerator for other companies.
When will OpenAI deploy Jalapeño?
OpenAI has been developing the chip for large-scale deployment, with current reporting indicating limited deployment by the end of 2026 and broader scaling planned for 2027.
Does Jalapeño replace Nvidia GPUs?
No. Jalapeño is specialized for inference and does not replace the broader capabilities of Nvidia’s AI accelerators. OpenAI is expected to continue using Nvidia hardware alongside its own custom silicon.
Why is OpenAI building its own AI chip?
A custom chip can give OpenAI greater control over inference performance, energy efficiency, hardware optimization and the cost of serving AI models at very large scale.
