> I tried kernel autoreasearch using DeepSeek-V4-Flash as well. It spent about 1-2 hours to complete the FlashAttention optimization job

Doing the same, re-implementing a lot of LLM/diffusion models in Rust+CUDA for my own usage, usually the initial implementation takes 1-2 days (of 100% autonomous work) then I put an agent to optimize the implementation which tends to get close to SOTA performance within another day or two.

As long as you can point the agent at "This is the correct baseline, make sure any optimizations still pass this", seemingly you can leave them and they come back after N hours with a faster program that just works.