>"CLM-8B also sets a new SOTA on challenging agentic coding benchmarks, including DeepSWE (81.6%) and Terminal Bench 2.1 (87.6%). [...]

A full pre-training run on the Nemotron DQA dataset takes about an hour on a single RTX 4090 GPU.

Most importantly, since states and actions are disaggregated, their embeddings can be cached independently. In settings where the state evolves continuously (e.g., Super Mario) while the action set remains fixed, we only need to recompute the state embedding at each step and can reuse the cached action embeddings. This substantially reduces inference cost, with the efficiency gains becoming increasingly significant as the number of candidate actions and context length grows."

There does definitely seem to be something there with respect to Contrastive Language Models.

They are probably worth studying for people (like myself!) who want to wring the absolute last cycle of local AI training and inferencing performance out of consumer-grade (i.e., not datacenter scale nor cost) hardware...