Post
22
Our paper is out on arXiv: how we trained Zuse, a decision model, across 27 European languages.
Decision models answer a fixed question about a document (yes/no, which category) with a probability, instead of generating text. Most are trained and tested in English. We wanted to know how far you get in the other European languages.
What's in the paper:
- the training data: permissive public datasets plus our own generated and translated data, nothing from proprietary models
- a per-language table for all 27 languages, including where it's weak (Nynorsk is at 67%)
- calibration: expected calibration error 0.016 pooled (0.035 weighted per source) for the 27B
Paper: Zuse: A Decision Model for European Languages (2611.06238)
Try it: https://bineric.com/zuse
Decision models answer a fixed question about a document (yes/no, which category) with a probability, instead of generating text. Most are trained and tested in English. We wanted to know how far you get in the other European languages.
What's in the paper:
- the training data: permissive public datasets plus our own generated and translated data, nothing from proprietary models
- a per-language table for all 27 languages, including where it's weak (Nynorsk is at 67%)
- calibration: expected calibration error 0.016 pooled (0.035 weighted per source) for the 27B
Paper: Zuse: A Decision Model for European Languages (2611.06238)
Try it: https://bineric.com/zuse