AI Models Can Provably Hide Arbitrary Capabilities, and Evals Cannot Catch It

A paper accepted at NeurIPS 2026 establishes a result that quietly undermines a load-bearing assumption of AI safety evaluation: “AI Models Can Provably Hide Arbitrary Capabilities” (arXiv:2610.13928, by Andis Draguns, Sumeet Ramesh Motwani, Raymond Douglas, and Christian Schroeder de Witt) shows that models can conceal arbitrary capabilities so they are cryptographically hard to discover, even when the auditor has full access to the weights. Capability evaluations assume a capable evaluator can elicit any dangerous behavior that exists. This paper shows that assumption fails under adversarial conditions, and demonstrates it empirically.

The Mechanism: Encrypted Networks in the Weights

The method extends earlier work that could only hide fixed strings. Here, backdoor attacks embed an encrypted neural network inside a host model’s weights; the hidden network is decrypted and executed only when a secret trigger is present, and “digital lockers” provide the hiding mechanism, with security reduced to standard cryptographic assumptions. That is what “provably hard to elicit and interpret” means concretely: an evaluator who queries the model, inspects its weights, or fine-tunes it is not merely facing a hard search problem, but one whose difficulty is backed by cryptography. The authors’ proof-of-concept PyTorch implementations hide a chemical-reaction-prediction capability, a deliberate stand-in for CBRN-relevant skills.

The Defenses Tried, and Why They Failed

The empirical half matters because it closes the “in practice it won’t work” escape hatch. Four defenses were tested against the backdoor: fine-tuning, weight noising, gradient-based trigger search, and latent adversarial training. All resisted, and the cryptographic guarantees back the empirical results. The authors release their models to support defense research, which is the right move, but the headline for anyone running capability evaluations is now: passing an eval does not prove a model lacks dangerous capabilities, even with full weight access and white-box tooling. Every elicitation-based audit, from eval suites to red-team schedules, inherits this bound.

The Implications for the Evals-Based Safety Stack

The result lands on an industry that has bet heavily on evaluations as its primary safety instrument: Anthropic’s automated behavioral audits, Google’s pre-release access testing for Gemini 4 Argon, and the FTC’s probe into whether companies misled the public about product risks all assume evals reveal what models can do. The paper does not claim any specific deployed model hides capabilities; it proves the audit method cannot rule it out. That distinction is the whole safety-theoretic point, and it echoes the shared-blind-spot caution in Sakana’s MASS: both papers this week identify structural limits of self-evaluation, from opposite directions.

The Limitations, Stated by the Authors

The paper’s honest constraints: demonstrations are small-scale, not on frontier models; the proofs rest on cryptographic assumptions rather than unconditional security; and only four defense approaches were studied, leaving other detection strategies untested. Those are real gaps between proof-of-concept and deployed frontier systems, and the authors do not paper over them. But the direction of the result, from hiding fixed strings to hiding arbitrary computations, is the worrying trend line, and the defenses that failed are the standard ones.

What to Watch

Three things. First, scaling: whether the encrypted-network technique transfers toward frontier-scale models is the question that determines whether eval-based safety cases need revision or replacement. Second, defense research funded by this release: the authors published specifically to support improvements, and cryptographic detection methods (detecting the locker structure rather than eliciting the trigger) are the obvious next target. Third, the policy echo: AGMAI’s provenance checklist and eval-based regulatory frameworks both assume disclosure and testing reveal capabilities; this paper gives the first cryptographic argument for why that assumption needs a caveat, and regulators drafting AI Act technical-documentation requirements should read it before finalizing the forms.