BaseDecision-Dyno

A 4B-class decision model with calibrated option probabilities.

Give BaseDecision-Dyno a document, conversation, policy, or structured record—and questions with caller-defined options. It returns the selected option and a probability for every option. It scores decisions directly, without generating explanations or decoding an answer sequence.

The calibrated BF16 checkpoint scores 53.83 on the full public Decision Index 0.3 suite, using the official evaluation kit.

At a glance

Property	Release
Model family	Qwen3.5-4B
Modality	Text only
Weights	Standalone merged checkpoint; no adapter stacking required
Runtime	Custom Hugging Face Transformers AutoModel
Default inference	One scoring pass per question
Optional inference	Explicit mode="flash", single read
Output	Option key and normalized option probabilities
Choice options	2–255 through the evaluated Decision Index adapter
Context	16,384 rendered tokens per question, including prompt overhead
Precision	BF16 backbone; FP32 decision readout and softmax
Attention	Bidirectional full-attention layers; native recurrence in linear-attention layers
Generation / decoding cache	No autoregressive generation; decoding KV cache disabled

Quickstart

The release includes custom model code. Review it before enabling trust_remote_code=True. Use AutoModel and the structured decide() interface to preserve the trained prompt, attention, option mapping, and calibration.

The validated environment used Python 3.12.15, PyTorch 2.14.0+cu130, and Transformers 5.17.0. Package loading and sampled protocol checks also passed on an NVIDIA A100-SXM4-80GB. The examples below require a compatible NVIDIA CUDA GPU.

import torch
from transformers import AutoModel

MODEL_PATH = "onlyaady/BaseDecision-Dyno"

# Match the backend setting used by the evaluated engine.
torch.backends.cuda.enable_cudnn_sdp(False)

model = AutoModel.from_pretrained(
    MODEL_PATH,
    trust_remote_code=True,
    dtype=torch.bfloat16,
    device_map={"": "cuda:0"},
).eval()

request = {
    "state": "The customer's invoice contains the same charge twice.",
    "questions": {
        "team": {
            "type": "choice",
            "instructions": "Which team should handle this request?",
            "criteria": {
                "billing": "Charges, invoices, and refunds",
                "technical": "Technical integration problems",
                "sales": "Questions about buying the product",
            },
        },
        "duplicate": {
            "type": "noul",
            "instructions": "The customer was charged twice for the same item.",
        },
    },
}

# Read twice is the default. Calibration is applied internally.
result = model.decide(request)
assert result["read_twice"] is True

print(result["answers"]["team"]["choice"])
print(result["answers"]["team"]["probabilities"])
print(result["answers"]["duplicate"]["probabilities"]["true"])

# Explicit single-read mode, with its own calibration applied internally.
flash_result = model.decide(request, mode="flash")
assert flash_result["read_twice"] is False

For a single question, {"state": ..., "question": ...} is also accepted; its answer is returned under answers["q"]. Multiple questions are processed independently, each in its own sequence and forward pass. They do not share a single prefill pass.

The response includes answers, mode, read_twice, sequence_tokens, calibrated, and release_id. For both decision types, the native API returns choice and probabilities. A noul answer uses the probability keys "true" and "false"; the Decision Index adapter converts this distribution into the benchmark’s scalar noul response.

Persistent inference

Download the model and its included JSONL runner:

hf download onlyaady/BaseDecision-Dyno --local-dir BaseDecision-Dyno
cd BaseDecision-Dyno

# Default: read twice, with calibration applied internally.
python inference.py --input requests.jsonl

# Explicit single-read mode, with its own calibration.
python inference.py --mode flash --input requests.jsonl

Each input line is one structured JSON request; each output line is one JSON response. Create requests.jsonl with your requests before running these commands.

The runner loads the model once and keeps it resident across successive requests. Omit --input to read requests from stdin until EOF. Use the validated CUDA environment. No compilation, quantization, or cross-request batching is enabled by this runner.

Read twice by default

Read twice repeats the state and question/options content within one rendered sequence, followed by one scoring pass per question. It is not two independent predictions, voting, or generated reasoning.

• model.decide(request), model(request), and model.prepare_request(request) default to read twice.
• mode="flash" explicitly selects a single read.
• The 16,384-token limit applies after rendering and repetition. Default mode therefore accepts less original content than flash mode.
• Oversized requests raise a length error. Inputs are not silently truncated, and default mode never falls back automatically to flash.
• Keep state and questions structured; do not manually repeat the input before calling decide().

The default is implemented in the supplied model code rather than left to the caller's prompt. Someone who modifies the code or directly uses lower-level weights can change that behavior.

Decision Index results

Completed self-run public evaluation with the official kit, edition 0.3. The exact calibrated model and strict default protocol above were used. These are public results, not a maintainer-run private or Full leaderboard score.

Metric	Result
Public Decision Index	53.83
Raw index	65.19
Breadth skill score	52.16
Requests processed before official exclusions	140,620
Requests after official exclusions	140,178
Successful scored requests	140,016
Unsupported scored requests	162
Errors	0

Index values are scores, not accuracy percentages. The public composite includes 37 panel benchmarks. The official report also contains additional diagnostic benchmark entries.

Area	Skill score (0–100)	Raw score (0–100)
Knowledge & Reasoning	35.96	50.26
Language Understanding	62.37	73.23
Retrieval & Classification	59.15	70.28
Tools & Automation	71.66	74.19
Arts & Human Taste	34.70	56.36

Raw scores combine benchmark-specific metrics; they should not all be read as plain accuracy. See the accompanying scores.json for individual benchmarks, coverage, and scoring details.

Unsupported requests were retained in the official scoring path: POP909-CL (158), ToolRet (2), API-Bank (1), and HLE (1). The summary reports alone do not establish the individual refusal reason for every row.

Calibration

Mode-specific scalar temperatures are applied once to raw logits, followed by a softmax over the current question's options:

probabilities = softmax(raw_logits / temperature)

Mode	Temperature
Default / read twice	2.0561549881751153
Flash / single read	1.9163401529278457

decide() already applies the appropriate temperature. Do not apply another temperature to its returned probabilities. These values replace the earlier 1.26 setting.

Temperatures were fitted on 10,000 decisions and audited on 10,000 separate decisions: 8,000 held out by group and 2,000 from families absent from temperature fitting. Those families were not necessarily absent from backbone training.

Mode	Audit metric ↓	Earlier T = 1.26	Calibrated
Default	Negative log-likelihood	0.897542	0.817370
Default	Brier score, summed over options	0.390352	0.365954
Default	15-bin hard-label ECE	0.107526	0.028381
Flash	Negative log-likelihood	0.876103	0.823770
Flash	Brier score, summed over options	0.385906	0.368202
Flash	15-bin hard-label ECE	0.091027	0.030229

Accuracy on the 9,266 hard-labeled audit decisions was unchanged by calibration: 70.26% in default mode and 69.78% in flash mode. Positive scalar temperature scaling preserves the selected option. These audit accuracies are separate from the Decision Index score.

Calibration improved on this audit distribution; it is not guaranteed on every domain. Training, quantization, or a material backend change requires renewed evaluation and may require recalibration.

Performance and deployment

The completed public run measured the following request latencies on shared NVIDIA RTX PRO 6000 Blackwell resources:

Default read-twice latency	Time
Median	53.3 ms

These are CUDA-synchronized individual-request timings from the public evaluation. Shared physical GPU contention and the benchmark request mix affect the measurements. They are not the maintainers' dedicated-hardware latency result, and they exclude model loading and external network/queueing overhead.

On an earlier short-input sample (60 distinct requests per mode, each measured three times; up to 512 original single-read rendered tokens and at most 32 options), the uncompiled runtime measured:

Mode	Median	p95
Default / read twice	29.58 ms	63.43 ms
Flash / single read	26.15 ms	36.32 ms

The short-input results do not establish a universal 150 ms default or 100 ms flash bound. Long contexts and many-question requests can take substantially longer.

Keep the model loaded across requests. The supplied runner does this without changing the scoring path. Compilation and quantized inference are not qualified for this release. A 16GB laptop deployment remains a target; this BF16 GPU release does not establish laptop memory fit or latency.

Limitations

• Text-only release; image, audio, and video inference are not supported.
• The evaluated adapter supports choice and noul; a dedicated ordinal score response interface is not qualified by this release.
• Default repetition consumes context and compute. The context limit is per rendered question, not 16k original tokens plus a second copy.
• Option probabilities are normalized over the supplied options. They do not prove that the option set contains a correct answer.
• English-focused evaluation does not establish comparable multilingual performance.
• Factual, numerical, ambiguous, and out-of-domain decisions can be wrong, including when confidence is high.
• Adversarial robustness and resistance to instructions planted in the state have not been established.
• Owners can modify the supplied runtime; read-twice enforcement is an API behavior, not a cryptographic restriction on weights.

Licensing and acknowledgements

This release builds on Qwen3.5-4B. Thanks to the maintainers of Tasksource.

Qwen3.5-4B's base license does not by itself settle the licensing of this derived release or all training sources. Preserve applicable upstream notices and dataset terms. A standalone distribution license and commercial-use clearance are not established by this card. No license badge is assigned in the metadata until those terms are resolved.