even in pre-training, usually 30%-50% is code these days.
That would be far too high in my opinion. But happy if anybody can give insights from their own experience with pretraining runs.
alt Hacker News
That would be far too high in my opinion. But happy if anybody can give insights from their own experience with pretraining runs.