alt Hacker NewsThis article does two things which make it unhelpful for me:
1. It uses math notation. This is a software development site, wouldn't this be much more clear to this audience written as psuedo code instead of a complex series of greek and latin letters?
2. It doesn't step through a concrete example. What is hollow R raised to the (M * d_v) ? Is it a number, a set, a list, a matrix?
To use an example I understand: RAG used to seem complicated to me when only math people were talking about it in math words. For example, let's take a line from wikipedia on embeddings:
> the representation is a real-valued vector that encodes the meaning of the word in such a way that the words that are closer in the vector space are expected to be similar in meaning
What made it clear was learning that generating an embedding is just a function(string) -> list[float], and RAG is just taking a list[float] for your prompt and finding other list[float] that are close to it in your dataset.
So what would make this article more clear, is step through a simplified example of putting a value into a memory store and querying that memory store, showing the actual data structures.
Replies
FWIW, I found the idea of interference-to-signal ratio as a way to think about memory capacity to be intuitive and was really interested in getting a better sense of how models increase capacity, until it hit the first math notation formula. My disconnect was based on the approachable tone of the intro, from which I assumed it would be a more generally accessible article.
To be clear, I have no problem with math people conveying math stuff with math notation but it seemed like the type of article which, even if it will get into notation, first at least explains "the main idea of it" without notation, then expands with notation as necessary when fleshing details or implementation.
Wait, you're saying that mathematics isn't useful to software engineers?
You should give it a go.
I guess you're not the target audience for this article then.
This is pretty approachable maths, and explain really well. I've seen far worse in programming language theory papers, which would qualify as "software development" in my mind. There are plenty of users of this forum that would find this paper useful and interesting, with sufficient math background to read it to boot.
Those are valid bits of feedback to make an articel more accessible, but it's totally fine and apprpriate as it is
What made it not useful to you likely makes it much more useful for me.
Fortunately, LLMs can work wonders in adapting content to be more accessible to a wide variety of audiences. Just give it a link and feed your exact post to it. It will be able to translate it into pseudocode and provide examples.
The author has a link to the actual code behind the blog post. https://github.com/KeAWang/associative-memory-blog/blob/main...
Well, you could tell your AI of choice to read the article and explain the math. I did that with ChatGPT and it helpfully translated a lot of it into Python for me.
Upvoting you for this is quite fair. Mathematicians do seem to suffer from the Feldspar Problem:
>It's easy to forget that the average person probably only knows the formulas for Olivine and one or two Feldspars
That being said the llm is indeed the universal translator — the babelfish supreme — and is excellent at translating whatever you want into pseudocode in your language of choice.
Send the link to Claude or your favorite alternative, and cut-and-paste your comment into the box, and you'll get exactly what you want.
[dead]
> This is a software development site, wouldn't this be much more clear to this audience written as psuedo code instead of a complex series of greek and latin letters
At the risk of stating the obvious, this is a forum where people can post arbitrary links to other sites, some of which might have audiences that might not be "the exact set of people who read Hacker News". I'm not sure you'd be able to make a compelling argument that alexwang.ai is aimed at software developers and no one else.