There's quite a bit more to culture and language than just being able to have transformers come up with believable language and/or dialect.

The point is that norway willl have its own LLM. And will not have dependencies to another state or private company. The goal is not to be the best model. But to have a model that include more Norwegian data then other LLM and that it's not screwed against other sources.

Yes transformers are great at translation as that is their purpose.

LLMs are not great at preserving cultural uniqueness and diversity. Take how “delve” has reentered the lexicon because the human assessors for pre training dialect of English uses “delve” a lot.

There is a lot of benefits to training specifically for a unique culture with unique norms to preserve the culture as we increasingly rely on LLMs.

https://www.scientificamerican.com/article/chatgpt-is-changi...

Definitively not the case in distilled models.

Model can speak Lithuanian too, but with a Russian accent which is a big taboo for us.

They're only good at it because they were trained on massive amounts of English and French data.