Depth Recurrent Vector
Qwen3-1.7B continued-pretrained with layers 8-19 repeated (shared weights): pass counts 2-5, flexible depth, ablations, no-loop control
Text Generation • 2B • Updated • 259Note Reference: Control: same data and recipe, no looping (a standard Qwen3-1.7B).
CharlesDDDD/DRV-Qwen3-1.7B-loop3-avg3
Text Generation • 2B • Updated • 277Note Main: Recommended. Block repeated 3 times; average of the weights of 3 independent training runs.
CharlesDDDD/DRV-Qwen3-1.7B-loop3-run1
Text Generation • 2B • Updated • 271Note Main: Block repeated 3 times (single training run).
CharlesDDDD/DRV-Qwen3-1.7B-loop3-run2
Text Generation • 2B • Updated • 270Note Main: Block repeated 3 times; exact repeat of `loop3-run1` (same settings, same random seed).
CharlesDDDD/DRV-Qwen3-1.7B-loop3-run3
Text Generation • 2B • Updated • 270Note Main: Block repeated 3 times; same settings as `loop3-run1` with a different random seed.
CharlesDDDD/DRV-Qwen3-1.7B-loop2
Text Generation • 2B • Updated • 275Note Main: Block repeated 2 times.
CharlesDDDD/DRV-Qwen3-1.7B-loop4-run1
Text Generation • 2B • Updated • 276Note Main: Block repeated 4 times.
CharlesDDDD/DRV-Qwen3-1.7B-loop4-run2
Text Generation • 2B • Updated • 281Note Main: Block repeated 4 times; different random seed from `loop4-run1`.
CharlesDDDD/DRV-Qwen3-1.7B-loop5
Text Generation • 2B • Updated • 275Note Main: Block repeated 5 times.
CharlesDDDD/DRV-Qwen3-1.7B-loop-random234
Text Generation • 2B • Updated • 274Note Flexible depth: Trained with a random number of passes each step (2, 3 or 4 with probability 20/60/20%).
CharlesDDDD/DRV-Qwen3-1.7B-loop-random234-uniform
Text Generation • 2B • Updated • 282Note Flexible depth: Trained with a random number of passes each step (2, 3 or 4, equally likely).
CharlesDDDD/DRV-Qwen3-1.7B-loop3-then-random
Text Generation • 2B • Updated • 286Note Flexible depth: Trained with 3 passes for the first 80% of training, then a random 2, 3 or 4 passes.
CharlesDDDD/DRV-Qwen3-1.7B-loop4-avg2
Text Generation • 2B • Updated • 267Note Main: Block repeated 4 times; average of the weights of `loop4-run1` and `loop4-run2`.
CharlesDDDD/DRV-Qwen3-1.7B-loop3-loop4-avg
Text Generation • 2B • Updated • 267Note Flexible depth: Average of the weights of `loop3-run1` (3 passes) and `loop4-run1` (4 passes).
CharlesDDDD/DRV-Qwen3-1.7B-ablation-loop3-final-loss-only
Text Generation • 2B • Updated • 125Note Ablation: Like `loop3-run1`, but trained with the loss on the final pass only.
CharlesDDDD/DRV-Qwen3-1.7B-ablation-loop3-chained
Text Generation • 2B • Updated • 120Note Ablation: 3 passes with a different way of connecting passes: each pass reads the previous pass's output instead of returning to the block's original input.
CharlesDDDD/DRV-Qwen3-1.7B-ablation-loop3-chained-final-loss-only
Text Generation • 2B • Updated • 122Note Ablation: Like `ablation-loop3-chained`, trained with the loss on the final pass only.