NVIDIA has released two open tools to lower the cost of running long-running AI agents. The release includes Nemotron 3.5 Lightning, a 30-billion parameter model with only 3 billion active parameters, and NeMo Switchyard, an open-source library that directs agent workflow steps to the most efficient model available. The combination addresses a specific bottleneck: agents spend most of their time on tool calls, result validation, and subagent delegation. Sending every step to a frontier reasoning model increases cost and latency. Lightning is built on a hybrid Mamba-2, MoE, and Attention architecture with a 1 million token context window. NVIDIA reports output speeds up to four times faster than similar models and a 30% reduction in completion time for 10,000 PinchBench tasks compared to Qwen3.6 35B at comparable accuracy. Companies including CrowdStrike, Harvey, CodeRabbit, Fastino Labs, and Lila Sciences are already customising it for cybersecurity, legal, coding, finance, and healthcare workloads.
Is it deployable?
Yes. Nemotron 3.5 Lightning is generally available under the permissive OpenMDW-1.1 license. NVIDIA states the model is ready for commercial use. The release includes open weights, training data, and recipes.
- Which companies: Anyone with a single modern GPU. NVIDIA lists single-GPU deployment on 1x DGX Spark (GB10) or 1x H100. That puts solo developers and seed-stage startups on the same footing as enterprises. Mid-market teams can serve it from Baseten, Together AI, or Nebius; regulated enterprises can keep it fully on-premises.
- Industries: Cybersecurity, legal services, software engineering, financial services, healthcare, and life sciences all appear in NVIDIA’s named customer set.
- Applications: Tool calling, result validation, subagent delegation, code review routing, log triage, contract parsing, and long-context retrieval across a 1M-token window.
The execution layer, not the planning layer
Long-running agents spend most of their time on high-volume execution. Tool calls, result validation, and subagent delegation dominate the token budget. Routing every one of those steps to a frontier reasoning model adds cost and latency.
Nemotron 3.5 Lightning targets that execution layer. It is a 30B mixture-of-experts model with 3B active parameters, built on a hybrid Mamba-2 + MoE + Attention architecture. Context length reaches 1M tokens. Pre-training covered more than 20 trillion tokens using an NVFP4 recipe.
The model is the smallest member of the Nemotron 3 family. Frontier models such as Nemotron 3 Ultra handle orchestration and planning, while Lightning handles the routine calls beneath them.
Where the speed comes from
Two mechanisms drive the performance.
- Speculative Decoding: Multi-token prediction was baked in during a dedicated pre-training stage, then improved with an MTP-boosting phase. NVIDIA also ships two external draft models: DSpark, a semi-autoregressive drafter recommended for DGX Spark and low-concurrency data center workloads, and DFlash, which uses a lightweight block-diffusion model.
- Quantization: An NVFP4 checkpoint ships alongside BF16. The same checkpoint serves Blackwell and Hopper natively, and extends to Ampere through W4A16 kernels.
NVIDIA reports up to 4x output speed versus similar-sized models. On PinchBench, it reports 86% accuracy while completing 10,000 tasks 30% faster than Qwen3.6 35B at comparable accuracy.
Published model card results (BF16 / NVFP4) show MMLU Pro 81.94 / 81.62, GPQA Diamond 75.44 / 75.57, SWE-bench Verified 51.56 / 52.80, Terminal-Bench 2.1 24.58 / 23.46, and AA-LCR 52.00 / 49.19. Recommended sampling is temperature 1.0 and top_p 0.95.
NeMo Switchyard
NeMo Switchyard is an open source library that routes each step of an agent workflow to the most capable and efficient model available.
It offers tuning-free routers, including an LLM classifier with session affinity, a stage router that reads recent tool activity, and an escalation router that starts cheap and promotes on sustained difficulty. A tunable prefill router learns from the model’s residual stream to predict which candidate will succeed. The reference server accepts OpenAI, Anthropic, and Responses API requests.
Two published results demonstrate the routing logic. LangChain benchmarked 145 multi-turn agentic tasks. Routing between Lightning and Claude Opus 4.8 with the escalation router cut cost 74% versus a frontier-only baseline, sending 7% of calls to the frontier model, at a roughly 6-point accuracy tradeoff. Cognition implemented staged routing in Devin Desktop. On FrontierCode Main, routing between Opus 5 and Kimi K2.7 reached 50.6% at a $3.11 mean cost, within 2.8 points of Opus 5 accuracy at approximately 28% lower mean cost.
How it affects people making things
Developers building agents no longer need to pay premium rates for every single step an agent takes. The new tools allow a workflow to start with a cheap model for routine tasks like code review or log triage, only escalating to a heavy model when the task becomes difficult. This lowers the barrier for startups to run complex, multi-step agents that were previously too expensive to sustain.
What it means
The release separates the cost of reasoning from the cost of execution. By providing a fast, open model for the bulk of agent work and a router to manage the mix, NVIDIA offers a practical path to reducing inference spend without sacrificing capability on routine tasks.
