QAH Pushes 4-Bit Hypernova-60B Past bfloat16 Source

Multiverse Computing’s Quantization-Aware Healing paper, released on August 25, 2026, describes a 4-bit model that outperforms the bfloat16 checkpoint used as its compressed source. The resulting Hypernova-60B uses about four times less weight memory and half as many parameters as the original gpt-oss-120b model.

The result comes from Quantization-Aware Healing (QAH), a post-training recovery method for models subjected to structural compression and low-precision quantization. Multiverse Computing applied it to OpenAI’s gpt-oss-120b, reducing the model from 117B parameters to roughly 60B through CompactifAI, then quantizing the remaining weights to MXFP4 4-bit precision.

Direct distillation from the original model

Compound compression creates two separate sources of quality loss. Structural compression removes layers, attention heads, or neurons, while quantization represents the surviving weights with fewer bits. Applying both can degrade reasoning, coding, mathematics, and long-context behavior more sharply than either operation alone.

Conventional quantization-aware training fine-tunes the compressed student with simulated low-precision forward passes. Quantization-aware distillation can improve that process by teaching the student to match a structurally compressed, full-precision checkpoint.

QAH changes the teacher. Instead of distilling from the intermediate bfloat16 compressed model, it uses the original uncompressed full-precision model as a frozen teacher. The student and teacher have different architectures, preventing direct parameter or hidden-state alignment. QAH therefore matches output distributions, an architecture-agnostic form of knowledge distillation.

The underlying rationale is important for compression pipelines. A structurally reduced model is generally produced from an existing checkpoint rather than independently pretrained from scratch. Its intermediate bfloat16 weights can therefore encode compression damage before quantization begins, imposing a quality ceiling on later recovery. Direct supervision from the original model gives the 4-bit student access to a stronger target.

Benchmark results

The evaluated student contains 59B total parameters and 4.8B active parameters, reflecting the Mixture-of-Experts design of gpt-oss-120b. QAH matched or exceeded its bfloat16 compressed source on 7 of 9 benchmarks.

Measurement	QAH result
Benchmarks matching or beating bfloat16 compressed source	7 of 9
Long-context reasoning gain	+7.4 points
Competition math gain	+5.6 points
LiveCodeBench comparison	Matched full, uncompressed 120B base model
Weight memory	Approximately 4× lower
Total parameters	Approximately 50% lower

The LiveCodeBench result provides the strongest reference point against the original model: the compressed MXFP4 checkpoint matched the full, uncompressed 120B base model on that benchmark. The broader gains, however, are reported against the bfloat16 structurally compressed source, not universally against the untouched 120B model.

Training speed and stability

QAH reached peak performance about seven times faster than a matched QAT baseline. It also avoided the catastrophic training collapse observed with QAT, allowing continued training without hand-tuned early stopping.

This distinction matters operationally. A recovery run that requires frequent early-stopping decisions is difficult to standardize across model sizes, compression ratios, and evaluation suites. Stable training reduces the need for manual intervention, although teams still need to validate the final checkpoint on their own workloads.

The technique complements existing work on quantization in AI and large-model distillation, while the CompactifAI pipeline is also available through an LLM compression API. Developers deploying sparse models can pair this result with established Mixture-of-Experts concepts when estimating active compute separately from total memory.

Model availability

Multiverse Computing released the resulting Hypernova-60B-2605 checkpoint under the Apache 2.0 license on Hugging Face. The QAH technique is covered by Patent Application Number 26382838.6 / P202602102EP.

If your deployment plan combines pruning or structural reduction with 4-bit inference, evaluate direct distillation from the original full-precision teacher rather than treating the compressed bfloat16 checkpoint as the final target. Track memory, active parameters, recovery speed, and task-level quality separately, because a lower-memory student can outperform its compressed source without matching the untouched base model on every benchmark.