README
🚀
🔄 In a Training Loop
Ash W.
Hoglet-33
AI & ML interests
Open source AI, datasets, parameter efficiency, SLMs, AI for the betterment of humanity. Contact at ash@basicallyai.co
Recent Activity
updated a Space about 23 hours ago
basically-ai/README reacted to Banaxi-Tech's post with 👀 1 day ago
We're delaying BananaMind 2.1!
When BananaMind 2.1 Lite was almost done, we benchmarked it and the results we're worse than BananaMind 2 Mini.
We're going to spend alot more time in research on tiny models and then scaling up our techniques to the actual BananaMind 2.1 models!
We're also announcing these new models:
BananaMind 2.1 Coder: A 149M instruction tuned coder model trained on 75B tokens + 10B tokens of stack-v3-train.
BananaMind 2.1 Pico: A 1M parameter model trained on 22B tokens of data.
We also may release BananaMind 2.1 Large with around 100M parameters depending on how much compute we have.
Please give us a follow!
https://huggingface.co/BananaMind
@Banaxi-Tech
---
@vovaRL
@DedeProGames
repliedto their post 3 days ago
We are announcing the first generation of the Pebble model family!
These are the models we are releasing:
- Pebble 10M
- Pebble 25M
- Pebble 50M
Each model will use a Mamba-Transformer 3:1 hybrid architecture and will be pretrained on 25 billion tokens before IFT and SFT.
Depending on development time and resources, we may also release:
- Pebble 5M
- Pebble 75M
- Pebble 1M (possibly)
We hope you're excited and enjoy the models!
Follow for more:
@Hoglet-33
https://huggingface.co/basically-aiOrganizations
updated a
Space about 23 hours ago
reacted to Banaxi-Tech's post with 👀 1 day ago
Post
1628
We're delaying BananaMind 2.1!
When BananaMind 2.1 Lite was almost done, we benchmarked it and the results we're worse than BananaMind 2 Mini.
We're going to spend alot more time in research on tiny models and then scaling up our techniques to the actual BananaMind 2.1 models!
We're also announcing these new models:
BananaMind 2.1 Coder: A 149M instruction tuned coder model trained on 75B tokens + 10B tokens of stack-v3-train.
BananaMind 2.1 Pico: A 1M parameter model trained on 22B tokens of data.
We also may release BananaMind 2.1 Large with around 100M parameters depending on how much compute we have.
Please give us a follow!
BananaMind
@Banaxi-Tech
---
@vovaRL
@DedeProGames
When BananaMind 2.1 Lite was almost done, we benchmarked it and the results we're worse than BananaMind 2 Mini.
We're going to spend alot more time in research on tiny models and then scaling up our techniques to the actual BananaMind 2.1 models!
We're also announcing these new models:
BananaMind 2.1 Coder: A 149M instruction tuned coder model trained on 75B tokens + 10B tokens of stack-v3-train.
BananaMind 2.1 Pico: A 1M parameter model trained on 22B tokens of data.
We also may release BananaMind 2.1 Large with around 100M parameters depending on how much compute we have.
Please give us a follow!
@Banaxi-Tech
---
@vovaRL
@DedeProGames
- 7 replies
replied to their post 3 days ago
Do these micro models have a specific use case? I mean they probably aren't going to be making code or doing general conversation... probably...
The point of Pebble isn't really to make tiny models that can compete with big LLMs. A big part of why people work on SLMs is to see how much capability you can actually get out of a very small model, and to experiment with different architectures, training methods, and ways of making models more efficient.
So, they probably aren't going to be amazing at everything, but that's not really the point. Pebble is mainly a research project to explore what is possible with models this small and hopefully contribute something useful to the SLM space. Though if you do want to see a coder model, I suppose I could make Pebble Coder at some point.
replied to Banaxi-Tech's post 3 days ago
Awesome!
reacted to Banaxi-Tech's post with 🔥 3 days ago
Post
2124
We have updated the BananaMind Base Bench leaderboard!
We now have these benchmark cards, they make it way easier to see which models are actually good!
We've also added the model advisor. It asks you what you want to use the model for and the parameter range and gives you the best model for your task!
Try it out at BananaMind/BananaMindBench-Leaderboard
And please give us a follow to BananaMind!
BananaMind
@Banaxi-Tech
We now have these benchmark cards, they make it way easier to see which models are actually good!
We've also added the model advisor. It asks you what you want to use the model for and the parameter range and gives you the best model for your task!
Try it out at BananaMind/BananaMindBench-Leaderboard
And please give us a follow to BananaMind!
@Banaxi-Tech
- 1 reply
replied to their post 3 days ago
They aren't released yet, but hopefully I can release them by the end of September though.
reacted to their post with 🔥 4 days ago
Post
2964
We are announcing the first generation of the Pebble model family!
These are the models we are releasing:
- Pebble 10M
- Pebble 25M
- Pebble 50M
Each model will use a Mamba-Transformer 3:1 hybrid architecture and will be pretrained on 25 billion tokens before IFT and SFT.
Depending on development time and resources, we may also release:
- Pebble 5M
- Pebble 75M
- Pebble 1M (possibly)
We hope you're excited and enjoy the models!
Follow for more:
@Hoglet-33
basically-ai
These are the models we are releasing:
- Pebble 10M
- Pebble 25M
- Pebble 50M
Each model will use a Mamba-Transformer 3:1 hybrid architecture and will be pretrained on 25 billion tokens before IFT and SFT.
Depending on development time and resources, we may also release:
- Pebble 5M
- Pebble 75M
- Pebble 1M (possibly)
We hope you're excited and enjoy the models!
Follow for more:
@Hoglet-33
- 7 replies
posted an update 4 days ago
Post
2964
We are announcing the first generation of the Pebble model family!
These are the models we are releasing:
- Pebble 10M
- Pebble 25M
- Pebble 50M
Each model will use a Mamba-Transformer 3:1 hybrid architecture and will be pretrained on 25 billion tokens before IFT and SFT.
Depending on development time and resources, we may also release:
- Pebble 5M
- Pebble 75M
- Pebble 1M (possibly)
We hope you're excited and enjoy the models!
Follow for more:
@Hoglet-33
basically-ai
These are the models we are releasing:
- Pebble 10M
- Pebble 25M
- Pebble 50M
Each model will use a Mamba-Transformer 3:1 hybrid architecture and will be pretrained on 25 billion tokens before IFT and SFT.
Depending on development time and resources, we may also release:
- Pebble 5M
- Pebble 75M
- Pebble 1M (possibly)
We hope you're excited and enjoy the models!
Follow for more:
@Hoglet-33
- 7 replies
reacted to Banaxi-Tech's post with 😎 6 days ago
Post
2691
We're releasing Overfitter 1.0.
Its a completly useless overfitted model trained on 202 epochs of SWE Bench Verified, SWE Bench Pro, Terminal Bench 2.1, DeepSWE.
It gets 100% on SWE Bench Verified, 98.6% on SWE bench Pro, 100% on Terminal Bench 2.1 and 100% on DeepSWE!
BananaMind/Overfitter-1.0
Its a completly useless overfitted model trained on 202 epochs of SWE Bench Verified, SWE Bench Pro, Terminal Bench 2.1, DeepSWE.
It gets 100% on SWE Bench Verified, 98.6% on SWE bench Pro, 100% on Terminal Bench 2.1 and 100% on DeepSWE!
BananaMind/Overfitter-1.0
- 11 replies
upvoted an article 6 days ago
Article
Extreme Overtraining in Tiny Language Models
Banaxi-Tech
• • 12 replied to Banaxi-Tech's post 9 days ago
Can't Wait, I'm so excited!
reacted to Banaxi-Tech's post with 🔥 9 days ago
Post
3782
We're announcing our BananaMind 2.1 model series!
The models will include:
- BananaMind 2.1 Nano: 10M parameters with 60B tokens.
- BananaMind 2.1 Lite: 25M parameters with 40B tokens.
- BananaMind 2.1 Flash: 50M parameters with 55B tokens.
- BananaMind 2.1 Pro: 135M-145M parameters (still deciding) with 100B tokens.
These model will use a multi tower architecture (like BananaMind/BananaMind-2.1-Unified) with some more architectural changes.
BananaMind 2.1 Pro will probrably use 2 no output towers, instead of one!
We're currently training some experimental models based on this architecture to see its scaling!
Follow us:
BananaMind
@Banaxi-Tech
@vovaRL
@DedeProGames
bananamind-research-community
The models will include:
- BananaMind 2.1 Nano: 10M parameters with 60B tokens.
- BananaMind 2.1 Lite: 25M parameters with 40B tokens.
- BananaMind 2.1 Flash: 50M parameters with 55B tokens.
- BananaMind 2.1 Pro: 135M-145M parameters (still deciding) with 100B tokens.
These model will use a multi tower architecture (like BananaMind/BananaMind-2.1-Unified) with some more architectural changes.
BananaMind 2.1 Pro will probrably use 2 no output towers, instead of one!
We're currently training some experimental models based on this architecture to see its scaling!
Follow us:
@Banaxi-Tech
@vovaRL
@DedeProGames
- 10 replies
reacted to Enderchef's post with 🚀 10 days ago
Post
4088
Do you support the SLM(1M-150M parameter) community?
If so, join the SLM discord(https://discord.gg/BBYaERvvn), and give these orgs some follows!
BananaMind
SupraLabs
fromziro
AxiomicLabs
If so, join the SLM discord(https://discord.gg/BBYaERvvn), and give these orgs some follows!
- 3 replies
reacted to onekq's post with 🔥 13 days ago
Post
3228
Lots of attentions are now on GPU residual value. I find car analogy to be useful.
* Both new and used cars can do the same job (you can run the latest model on 6-year-old A100)
* New cars are more efficient (higher performance per power draw)
* It's not the year, but mileage and maintenance
The last point is undeveloped. We need Carfax and KBB for GPUs.
And there will be lemon GPUs.
* Both new and used cars can do the same job (you can run the latest model on 6-year-old A100)
* New cars are more efficient (higher performance per power draw)
* It's not the year, but mileage and maintenance
The last point is undeveloped. We need Carfax and KBB for GPUs.
And there will be lemon GPUs.
- 1 reply
replied to Banaxi-Tech's post 13 days ago
Do you think an ARM version will come at some point? This would be something I would put on a Raspberry pi.
reacted to onekq's post with ❤️ 16 days ago
Post
2762
This is an easy-to-remember pattern.
GLM and Kimi are in Beijing, DeepSeek and Qwen are in Hangzhou.
OpenAI and Anthropic are in SF, xAI and Meta are in the peninsula.
In both China and the Bay Area, token price of one area is ~1/3 of the other area.
GLM and Kimi are in Beijing, DeepSeek and Qwen are in Hangzhou.
OpenAI and Anthropic are in SF, xAI and Meta are in the peninsula.
In both China and the Bay Area, token price of one area is ~1/3 of the other area.
- 1 reply
upvoted an article 16 days ago
Article
DAVIDAU DELETED MY POST AND BANNED ME FOR ASKING ABOUT BENCHMARKS — HERE IS THE FULL ANALYSIS (9B + 27B + 390 MODELS)
DedeProGames
• • 15 liked a
dataset 20 days ago
reacted to Banaxi-Tech's post with 🔥 20 days ago
Post
1974
We're exited to announce BananaMind OS, our OS specically for running BananaMind models!
Its able to run BananaMind 2 Nano at 4 bit on only 7-8MB of ram, the 2 bit on 6MB of ram and the 8 bit version on 14MB of RAM!
It runs on a 486 or newer!
Check out this video and image running BananaMind 2 Nano 4 Bit on 9
MB of RAM and a emulated 486 in QEMU at ~1TPS!
We asked it: "What is the first letter of the alphabet?"
The response is:
"The first letter of the alphabet is:
- A.
"
And if you're asking because of the video, yes I am a arch btw.
Comment and like this post for a GitHub link and comment for adding other models!
Its able to run BananaMind 2 Nano at 4 bit on only 7-8MB of ram, the 2 bit on 6MB of ram and the 8 bit version on 14MB of RAM!
It runs on a 486 or newer!
Check out this video and image running BananaMind 2 Nano 4 Bit on 9
MB of RAM and a emulated 486 in QEMU at ~1TPS!
We asked it: "What is the first letter of the alphabet?"
The response is:
"The first letter of the alphabet is:
- A.
"
And if you're asking because of the video, yes I am a arch btw.
Comment and like this post for a GitHub link and comment for adding other models!
- 47 replies
published a
Space 24 days ago