Papers
arxiv:2611.21421

AuXSpec: Learning to Autoformalize Program Specifications via Executable Postconditions

Published on Nov 16
Β· Submitted by
Haocheng Wang
on Nov 18
Authors:
,
,
,
,
,
,

Abstract

Formal specifications define intended program behavior and underpin verification, but writing them in theorem provers such as Lean is slow and expertise-intensive, making specification a major bottleneck to scaling formally correct code. Autoformalization from natural-language problems is promising, yet training specification models requires scalable feedback on whether generated specifications capture intended behavior. Prior approaches obtain this feedback through explicit proofs or separately implemented code with formal correspondence proofs, both expensive. We introduce AuXSpec, a Lean framework that pairs declarative postconditions with executable counterparts, obtained through type class resolution when possible and otherwise defined separately with an equivalence proof. Test cases can then evaluate declarative specifications by execution, letting us use agreement as a reinforcement-learning reward. Using AuXSpec, we synthesize 3,037 LeetCode problems and expand training to 19.5K samples through expert iteration on Codeforces. Experiments show that training Qwen3.5-9B and Qwen3.8-27B with the execution-based reward improves SpecGen pass@4 on AuXSpec-Test 0.7% to 82.7% and 14.3% to 84.3%, and the 27B model outperforms all frontier models. On the out-of-domain Verina benchmark with its original prompts, both models improve 7.0% to 90.5% and 21.9% to 90.5% and surpass VeriSpecGen-30B-A3B, the prior state of the art among open-source specification models, on pass@1 (theirs 47.6% vs. ours 73.7% and 84.5%) and pass@4 (theirs 63.9% at pass@10 vs. ours 90.5%). We release the weights of the trained 9B and 27B models and the 3,037 problems with test cases, including the 300-problem AuXSpec-Test benchmark with expert review and correctness proofs for future evaluation. Models and data: https://huggingface.co/collections/eth-algopt/auxspec. Website: https://auxspec.org

Community

We introduce AuXSpec, a Lean framework that pairs each declarative postcondition with an executable postcondition, so test cases can check generated specifications by execution. πŸ” This unlocks execution-based reinforcement learning for formal specification generation.

πŸ› οΈ Trained on 19.5K SFT samples and then with RL on 13.6K tasks under this execution-based reward, AuXSpec-27B-RL improves SpecGen pass@4 over its base model (AuXSpec-Test 14.3% β†’ 84.3%, out-of-domain Verina 21.9% β†’ 90.5%) and outperforms all frontier models on our 300-problem benchmark AuXSpec-Test. The model also generates the fewest output tokens per response (1.8K vs. 9.7K for Claude-Opus-5).

πŸ€— Models, data and the benchmark: https://huggingface.co/collections/eth-algopt/auxspec

image

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2611.21421
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 2

Datasets citing this paper 1

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.21421 in a Space README.md to link it from this page.

Collections including this paper 1