dotlabs
🚀
2
Dot Labs
company
AI & ML interests
byte-level research • the dots company
Recent Activity
appvoid
updated a
Space about 12 hours ago
appvoid
updated a
model 1 day ago
Update README.md
#1 opened 8 days ago
by 
j0no12
Update README.md
#2 opened 8 days ago
by 
KlondikeDev
Update README.md
#3 opened 8 days ago
by j0no12
Join Us!
5
#1 opened 9 days ago
by 
appvoid
appvoid
updated a
dataset 9 days ago
appvoid
published a
dataset 9 days ago
Post
171
- 7 replies
appvoid
published a
model 10 days ago
Post
200
byte-level, deep networks, flow-matching, matryoshka...
Post
180
Much of the misunderstanding on loops and byte-level models comes from fear on inventing the wheel, not reinventing it
Post
150
How much should OpenAI offer for a name like Dot Labs?
- 1 reply
Post
174
Reality has not limits if you know how to shape it, one step at a time.
- 1 reply
appvoid
updated a
dataset 23 days ago
Post
88
How old were you when you discovered good AI agents favor the Strategy Design Pattern over any other?
- 1 reply
appvoid
published a
dataset about 1 month ago
Post
134
LLMS for edge devices?
What if we go the reliable route instead of the speed route?
What if we make it run on sbcs with few megabytes available?
That's the idea for the next model.
Keep in tune.
What if we go the reliable route instead of the speed route?
What if we make it run on sbcs with few megabytes available?
That's the idea for the next model.
Keep in tune.
Post
144
We trained a 10.9M byte-level recurrent Transformer on L3 and L6. (Loop 3 and Loop 6)
Yet L4/L5 improved too, L8 held up, and the L3→L6 gain grew during training.
Same weights. More compute. Better predictions.
This is a new architecture for effective compute after several steps beyond original training!
We mixed and matched components like time and mhc into an ouro-like byte-level language model and the result is BET, a byte-level step-elastic transformer that can run computation steps without significant degradation.
One of the coolest parts of this training was discovering how Gradient Descent decided to use the first layer as what we would consider a scratchpad! Totally destroyed for the decoder but somehow makes total sense for the next layer!
I believe looped-transformers are the future of edge computing and this is a first step towards it.
Blogpost: https://medium.com/@appvoidofficial/byte-level-elasticity-182fe2ed1d2f
appvoid/bet-10m
Yet L4/L5 improved too, L8 held up, and the L3→L6 gain grew during training.
Same weights. More compute. Better predictions.
This is a new architecture for effective compute after several steps beyond original training!
We mixed and matched components like time and mhc into an ouro-like byte-level language model and the result is BET, a byte-level step-elastic transformer that can run computation steps without significant degradation.
One of the coolest parts of this training was discovering how Gradient Descent decided to use the first layer as what we would consider a scratchpad! Totally destroyed for the decoder but somehow makes total sense for the next layer!
I believe looped-transformers are the future of edge computing and this is a first step towards it.
Blogpost: https://medium.com/@appvoidofficial/byte-level-elasticity-182fe2ed1d2f
appvoid/bet-10m
Post
3,962
We got gpt6 before gta6
- 15 replies