Dharma AI Topology Scheduling Hits 85% H100 Cluster Utilization

Dharma AI increased capacity on its high-performance computing orchestrator by focusing on physical hardware routing. Detailed in a new engineering blog post, the company pushed utilization on a 512-GPU NVIDIA H100 cluster from 52% to 85% purely through software changes. For infrastructure engineers, this highlights how standard orchestration software often masks physical hardware capacity, addressing the broader problem that half of global GPU capacity sits idle.

The efficiency leap relies on a transition from simple resource-count scheduling to Topology-Aware Job Sequencing (TAJS). Standard schedulers typically assign workloads using a first-in, first-out sequence, grabbing the next available compute node regardless of its physical location in the data center rack.

Defeating Hardware Fragmentation

Dharma AI identified that random job assignment creates physical fragmentation across the cluster. When free GPUs are separated by one or two nodes, training runs cannot leverage high-speed RDMA connections. This fragmentation effect left 20% of their test cluster idle but unusable for intensive distributed training.

The new TAJS model solves this through Interconnect-First Packing. The system maps incoming jobs to the network’s minimal spanning tree. Any workload requiring 8 to 16 GPUs is strictly bound within an NVLink domain. By preventing jobs from straddling non-NVLink boundaries across separate InfiniBand rails, the orchestrator eliminates the tail latency spikes that bottleneck communication.

The Scheduling Algorithm

To consolidate workloads, the update implements Strict Bin Packing with Preemption (SBPP). This logic dynamically pauses lower-priority research tasks to secure contiguous physical blocks for high-priority production runs.

Before assigning hardware, the orchestrator initiates a Pre-flight Topology Check. Rather than immediately deploying a workload to disparate nodes, the system imposes a calculated delay of up to 180 seconds to wait for a contiguous block of GPUs to become available. This intentional delay trades queue time for execution speed. If you regularly find GPU gaps in PyTorch 2.12, auditing your cluster’s assignment queue is a necessary first step.

Throughput and Performance Gains

The architectural shift produced measurable improvements in end-to-end training metrics on a RoCE v2 fabric.

Metric	Previous Configuration	TAJS Update
Hardware	512 H100 GPUs	512 H100 GPUs
Cluster Utilization	52%	85%
70B+ Model Throughput	Baseline	+22% TTP

The 22% increase in training tokens per second (TTP) for models exceeding 70 billion parameters directly reflects the reduced communication overhead between physically proximate nodes.

If you manage distributed AI infrastructure, audit your orchestrator’s awareness of physical rack topology. Prioritizing contiguous hardware allocation over immediate execution can unlock substantial compute capacity without requiring additional hardware procurement.