Model Addition Request

#7
by Banaxi-Tech - opened

lol thats lots

@CodeSoft Whats the maximum size for this? i might make a smaller G1-MINI distilled with logits later as 6.5B total is a bit large .

@CodeSoft Whats the maximum size for this? i might make a smaller G1-MINI distilled with logits later as 6.5B total is a bit large .

There isn't a formal maximum size, but it's probably around 250M - 300M parameters.

ok lol i will distil to like 100M dense or smth to make it fit on all the leaderboards

CodeSoft changed discussion status to closed

Sign up or log in to comment