I think you’re talking somewhat at cross purposes to the original article.

The points I take away are:

- Good optimization is difficult and slow work, hence expensive, but LLMs can do it so we should be able to afford it more often now.

- There’s always a risk of over-fitting to your specific problem, but if everyone is now making bespoke optimizations maybe that isn’t actually a problem.

You said:

LLMs are terrible at optimizing memory utilization. There is just too little training code that does it well and far too much that doesn't.

There’s probably something in that, but it can be mitigated by testing against a local benchmark. LLMs are good at iterating tirelessly and finding incremental improvements. And as noted above, it doesn’t necessarily matter if your benchmark isn’t fully general.