Another book which I would highly recommend for learning the math (or at least getting a very good idea of how it works) behind machine learning/transformers is "Why Machines Learn: The Elegant Math Behind Modern AI" by Anil Ananthaswamy. I read it this past year and I found it was extremely good at pacing and building up intuition to the "how and why" of most important math concepts and discoveries in ML that have lead to where we are today. For those who don't already have a fairly deep understanding of linear algebra, statistics, and multivariate calculus, this book will be a far more gentle introduction yet still has enough rigor for you to come away with something.