Hugging Face's logo

wuff-mann
/
MANN-GPT

Text Generation
Safetensors
English
Chinese
memory-augmented
bilingual
mann-gpt
Model card Files
xet
 Community

MANN-GPT v6

Memory-Augmented Autoregressive Language Model

记忆增强型自回归语言模型

Causal MANN · Global Memory v2 · Dynamic Thinking Network (DTN)
因果 MANN · 全局记忆 v2 · 动态思考网络（DTN）

Architecture Phase 1 Phase 2 Languages License

GitHub · Hugging Face · ModelScope

Overview / 模型简介

MANN-GPT v6 is a memory-augmented autoregressive language-model architecture built around causal MANN layers, a formally qualified Global Memory v2 path, and a Dynamic Thinking Network (DTN). This repository is the frozen architecture and code archive for the first formally qualified bilingual continuation line.

MANN-GPT v6 是一种记忆增强型自回归语言模型架构，其核心由 因果 MANN 层（causal MANN layers）、已完成正式资格验证的 Global Memory v2 全局记忆路径，以及 动态思考网络（Dynamic Thinking Network, DTN） 构成。本仓库保存首条完成正式资格验证的中英双语继续训练线路所对应的冻结架构与代码归档。

Stable architecture freeze / 稳定架构冻结版本: v6-mature-20260910
Canonical qualified parent / 当前正式合格父模型: Formal Bilingual Continuation Phase-1 · Seq512 · 20260909T133242Z
Long-context experiment / 长上下文实验: Phase-2 · Seq1024 · HOLD / not promoted（未晋升）

Core design / 核心设计

English 中文
Causal MANN 26 causal MANN layers form the main autoregressive computation path. 26 层因果 MANN 构成主要自回归计算路径。
Global Memory v2 Source-retention-weighted segment memory with a causal remote reader beyond the local relation window. 基于 SourceRetention 加权的分段全局记忆，并通过因果远程读取器访问局部关系窗口之外的信息。
DTN A qualified fixed cumulative depth-3 Dynamic Thinking Network is retained as part of the mature architecture. 正式合格版本保留固定累计深度 3 的动态思考网络。
Bilingual continuation The qualified continuation line uses a protected EN/ZH representation while adapting only a restricted Tail-MANN scope. 正式继续训练线路保护中英双语表示，仅适配受限的 Tail-MANN 范围。
Strict causality Qualification includes exact autoregressive causality checks and protected-state preservation. 资格验证包含严格自回归因果性检查与受保护状态保持验证。

Release Status / 当前状态

✅ Stable / 稳定 🧪 Experimental / 实验
Formal Bilingual Continuation Phase-1 · Seq512

Architecture-readiness loop (Stage2R) closed.
Formal bilingual continuation completed successfully.
Canonical stable checkpoint qualified.
Exact checkpoint loadback verified.

正式双语继续训练 Phase-1 · Seq512

架构就绪闭环（Stage2R）已完成。
正式双语继续训练成功完成。
当前标准稳定 checkpoint 已通过资格验证。
checkpoint 精确回载验证通过。		 Phase-2 · Seq1024 — HOLD

Global-Memory utility improved.
Deconfounded memory retention passed.
Optimization gain: +0.4017% < +0.5000%.
Q2 target attribution: 0.1897 < 0.2000.
Not promoted as the stable parent.

Phase-2 · Seq1024 — HOLD

Global Memory 整体效用有所提升。
去混淆记忆保持验证通过。
优化增益：+0.4017% < +0.5000%。
Q2 目标归因：0.1897 < 0.2000。
未晋升为新的稳定父模型。

The canonical Phase-1 checkpoint passed value qualification, Global-Memory causal-identity retention, exact Q2 preservation, fixed-D3 retention, strict autoregressive causality, protected-state exactness, optimizer-resume lineage, and exact checkpoint loadback.

当前标准 Phase-1 checkpoint 已通过价值验证、Global Memory 因果身份保持、Q2 精确保留、固定 D3 保持、严格自回归因果性、受保护状态精确性、优化器续训谱系，以及 checkpoint 精确回载等资格检查。

Architecture at a Glance / 架构概览 

flowchart LR
    A[Token IDs<br/>词元 ID] --> B[Embedding + Semantic Adapter<br/>嵌入 + 语义适配器]
    B --> C[26 Causal MANN Layers<br/>26 层因果 MANN]
    C --> D[Fixed Cumulative D3 DTN<br/>固定累计 D3 DTN]
    D --> E[RMSNorm]
    E --> F[Tied LM Head<br/>权重绑定 LM Head]

    subgraph L[MANN Layer / MANN 层]
      L1[Gated Causal Depthwise Local Conv<br/>门控因果深度局部卷积] --> L2[MemorySelector]
      L2 --> L3[q/k/v Causal Relation Network<br/>q/k/v 因果关系网络]
      L3 --> L4[ReLU² Normalized Connections<br/>ReLU² 归一化连接]
      L4 --> L5[Iterative Activation Propagation<br/>迭代激活传播]
      L5 --> L6[SwiGLU FFN]
    end

    subgraph GM[Global Memory v2 @ L24 / L24 全局记忆 v2]
      G1[SourceRetention Writer<br/>SourceRetention 写入器] --> G2[16-token Segment Slots<br/>16-token 分段槽位]
      G2 --> G3[Causal Remote q/k Reader<br/>因果远程 q/k 读取器]
      G3 --> G4[Local + β·Remote Propagation<br/>局部 + β·远程传播]
    end

The relation path is causal and uses q/k geometry with RoPE, ReLU² non-negative relation scores, and explicit normalization rather than a softmax attention router. Global Memory v2 adds source-retention-weighted segment slots and a remote causal reader outside the 256-token local relation window.

关系路径保持严格因果性，并采用带 RoPE 的 q/k 几何结构、ReLU² 非负关系分数以及显式归一化，而不是使用 softmax attention router。Global Memory v2 进一步加入由 SourceRetention 加权的分段记忆槽位，并通过远程因果读取器访问 256-token 局部关系窗口之外的信息。

Frozen Model Geometry / 冻结模型几何参数

Component / 组件 Mature v6 setting / v6 成熟版本设置
Vocabulary / 词表大小 50,259
Hidden size / 隐藏维度 1,536
MANN layers / MANN 层数 26
FFN hidden / FFN 隐藏维度 4,096
Model max sequence / 模型最大序列长度 2,048
Local relation window / 局部关系窗口 256
Relation heads / 关系头 8 query / 2 KV
Global Memory layer / 全局记忆层 L24
Global slot size / 全局槽位大小 16 tokens
Global-memory β 0.60
DTN paths / DTN 路径数 6
Qualified DTN policy / 合格 DTN 策略 Fixed cumulative depth 3 / 固定累计深度 3

Mature Continuation Contract / 成熟继续训练契约

During the formally qualified continuation line, the protected architecture remains fixed while only the L23–L25 Tail-MANN scope is adapted. The Global Memory reader, SourceRetention writer, bilingual protected embeddings/rows, historical v_proj, and DTN remain frozen. Causal-retention replay is used to prevent target-memory identity drift during continued learning.

在正式合格的继续训练线路中，受保护架构保持冻结，仅适配 L23–L25 的 Tail-MANN 范围。Global Memory reader、SourceRetention writer、中英双语受保护 embedding/rows、历史 v_proj 与 DTN 均保持冻结；继续学习过程中使用 causal-retention replay（因果保持重放），以避免目标记忆身份发生漂移。

Canonical Phase-1 contract / Phase-1 标准训练契约

Training item / 训练项 Setting / 设置
Context / 上下文长度 512
Updates / 更新步数 768
Gradient accumulation / 梯度累积 4
Trainable scope / 可训练范围 L23-L25
Tail learning rate / Tail 学习率 5e-6
Data mixture / 数据配比 40% EN / 40% ZH / 20% Code
Causal replay / 因果重放 every 8 updates, 2 records/event
Replay teacher / 重放教师 frozen qualified parent
Adaptive runtime / 自适应运行时 closed

Qualification Evidence / 资格验证证据

Canonical stable run / 标准稳定运行

Run: 20260909T133242Z
Verdict: FORMAL_BILINGUAL_CONTINUATION_PHASE1_SEQ512_QUALIFIED_PROMOTE_PHASE2_CONTEXT_RAMP

Metric / 指标 Result / 结果
Weighted optimization gain / 加权优化增益 +0.6746%
Fresh real composite / 新鲜真实复合指标 +0.4612%
Q2 Global-Memory gain / Q2 全局记忆增益 7.4755%
Q2 target attribution / Q2 目标归因 0.2093
Deconfounded Global-Memory replication / 去混淆全局记忆复现 ZH→ZH 6/6 · ZH→EN 6/6 · EN→ZH 6/6
Strict causality / 严格因果性 9/9 exact-zero prefix-logit difference
Final loadback max-diff / 最终回载最大差异 0.0

Checkpoint integrity / Checkpoint 完整性

Canonical checkpoint hashes / 标准 checkpoint 哈希 
formal_phase1_state.pt
ce1d9e7abf5d04e78f63bfa2a23bf3bee4bd1a1f3c940ddf4af18cb78e0928b8

optimizer_state.pt
bae2f679f27a0af7bdccdb016b25f48b4e762dd942a1ac84845e4e5294a365f2

Hugging Face checkpoint path / HF checkpoint 路径

checkpoint/formal_bilingual_continuation/phase1/20260909T133242Z/qualified

Repository Layout / 仓库结构 

MANN-GPT-v6/
├── src/
│   ├── base_architecture.py
│   │   └── Qualified base MANN-GPT v6 architecture
│   │       已通过资格验证的 MANN-GPT v6 基础架构
│   │
│   ├── global_memory_components.py
│   │   └── SourceRetention writer, segment slots & Global Memory v2 reader
│   │       SourceRetention 写入器、分段槽位与 Global Memory v2 reader
│   │
│   ├── global_memory_install.py
│   │   └── L24 installation path & causal intervention hooks
│   │       L24 安装路径与因果干预 hooks
│   │
│   └── dtn_policy.py
│       └── Qualified fixed cumulative D3 policy
│           已通过资格验证的固定累计 D3 策略
│
├── docs/
│   ├── MATURE_ARCHITECTURE_FREEZE_20260910.md
│   │   └── Design freeze, responsibilities, invariants & stable/experimental boundary
│   │       架构冻结、职责、不变量及稳定/实验边界
│   │
│   └── FORMAL_TRAINING_STATUS_20260910.md
│       └── Qualification state & Phase-2 HOLD diagnosis
│           资格状态与 Phase-2 HOLD 诊断
│
├── manifests/
│   └── v6_mature_20260910.json
│       └── Machine-readable release/checkpoint lineage
│           机器可读的版本与 checkpoint 谱系
│
├── experiments/
│   ├── formal_phase1_20260909/README.md
│   │   └── Qualified Phase-1 evidence / Phase-1 合格证据
│   └── formal_phase2_20260910/README.md
│       └── Seq1024 HOLD evidence; not a promoted parent
│           Seq1024 HOLD 证据；未晋升为父模型
│
└── scripts/
    └── publish_to_hubs.py
        └── Hugging Face & ModelScope publication helper
            Hugging Face 与 ModelScope 发布辅助脚本

Stable vs Experimental / 稳定版本与实验版本

Stable / 稳定版本

MANN-GPT v6 + Global Memory v2 + frozen-reader continuation policy + fixed cumulative D3 + qualified Phase-1 Seq512 checkpoint
MANN-GPT v6 + Global Memory v2 + 冻结 reader 继续训练策略 + 固定累计 D3 + 已合格 Phase-1 Seq512 checkpoint

Experimental — not promoted / 实验版本——未晋升

Seq1024 Phase-2 context ramp retained memory utility and strict causality, but Q2 target attribution fell below the pre-registered floor and optimization gain did not reach the qualification floor.
Seq1024 Phase-2 上下文扩展实验 保持了记忆效用与严格因果性，但 Q2 目标归因低于预注册门槛，同时优化增益未达到资格门槛，因此不作为新的稳定父模型。

Repositories / 项目仓库

Platform / 平台 Repository / 地址
GitHub https://github.com/Mr-wuff/MANN-GPT-v6
Hugging Face https://huggingface.co/wuff-mann/MANN-GPT
ModelScope https://www.modelscope.cn/models/WuFeiFan/MANN-GPT

License / 许可证

This project is released under the MIT License. See LICENSE for details.

本项目采用 MIT License 开源。详细条款请参阅 LICENSE。

MANN-GPT v6
Qualified memory-augmented autoregressive architecture · 正式资格验证的记忆增强型自回归架构

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
Text Generation
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support