410 Tokens/Sec Nemotron 3.5 Lightning Hits SageMaker JumpStart

On August 17, 2026, Amazon Web Services added NVIDIA Nemotron 3.5 Lightning to Amazon SageMaker JumpStart. This release targets the execution layer of high-volume, long-running AI agents. If you build personal assistants, financial processors, or cybersecurity triage systems, this model isolates the repetitive tool-calling and validation steps from heavy frontier reasoning models.

Architecture and Speculative Decoding

The model uses a hybrid Mixture-of-Experts (MoE) design containing 30B total parameters. During a forward pass, it activates only 3B active parameters. The network interleaves Mamba-2, MoE, and selective Attention layers to maintain context retrieval over a 1M token window.

NVIDIA engineered the inference pipeline around speculative decoding to reach approximately 410 tokens per second. The architecture relies on two dedicated draft models. DFlash handles high-throughput context processing, while DSpark uses causal grouped-query attention with a sliding window of 1024 tokens. This combination delivers four times the throughput of equivalently sized models.

Benchmark Results

The model prioritizes speed and accuracy in repetitive agent workflows. NVIDIA tested the architecture on the PinchBench evaluation, where it achieved 86% accuracy. Lightning completed 10,000 tasks 30% faster than Qwen3.6 35B.

The model also established strong baselines in coding and scientific reasoning. On SWE-bench Verified, Lightning scored 51.56 in BF16 and 52.80 using the quantized NVFP4 checkpoint. The model recorded 81.94 on MMLU Pro and 75.44 on GPQA Diamond without external tool access.

Ecosystem and Routing Integration

To support multi-model architectures, NVIDIA released NeMo Switchyard, an open-source model routing library. Switchyard enables developers to route complex planning tasks to frontier models like Nemotron 3 Ultra while delegating high-volume execution steps to Lightning.

The release includes optimized support for standard harness libraries. You can drop the model directly into environments running Hermes Agent, LangChain, CrewAI, or OpenClaw. Early adopters of the architecture include Uniphore, Boomi, Cadence, and Siemens. Thoughtworks reported a 75% preference for domain-adapted legal versions of the model during private previews.

Deployment and Hardware Requirements

Through SageMaker JumpStart, developers can deploy the model via the catalog UI or the Python SDK without manual infrastructure provisioning. The 3B active parameter count allows the model to run on single-GPU instances, including an NVIDIA H100 or A100.

NVIDIA also made the weights available across Hugging Face, Ollama, Baseten, and the Google Cloud Gemini Enterprise Agent Platform. The model operates under the OpenMDW License Agreement, version 1.1, which provides access to weights, training data, and recipes for local deployments on hardware ranging from NVIDIA RTX PCs to DGX Spark clusters.

If your agent infrastructure bottlenecks on latency or cost during tool execution, splitting the workload is now a hardware-level option. You can map your planning logic to a large model and push the iterative, high-volume validation loops to Lightning running on a single A100.