alt Hacker NewsPost like these always drives a motivation to understand how inference or what OP did.
For folks who are more familiar in this space, is there a good road map how to build and understand the intuition behind this? I’m comfortable in Java and Python and do vanilla SWE work.
Replies
Intuition: there are YouTube videos from 3blue1brown [1] and Welch Labs [2] on neural networks, LLMs, and related tech that are good.
Building: the videos/tutorials from Andrej Karpathy in Python [3] are considered to be very good at building and understanding the basics, from a small/toy neural network engine to transformers and tokenizers.
There was a post here a few weeks ago [4] on reasoning models using Python/PyTorch with links to an older series on non-reasoning transformers.
[1] https://www.youtube.com/playlist?list=PLZHQObOWTQDNU6R1_6700...
[2] https://www.youtube.com/@WelchLabs/videos
I'm not an expert but there's https://jaxformer.com, https://github.com/karpathy/nanochat and, of course, the original "Attention is all you need" paper at https://arxiv.org/abs/1706.03762.