Justin Lebar (he built xla compiler and worked at OpenAI) has an amazing talk about this subject https://youtu.be/cyJU32ivIlk?si=gYuHtzMJIvaSqcht