logoalt Hacker News

netruk44 • yesterday at 2:09 PM • 7 replies • view on HN

I’ve been playing around with mapping steam games + their reviews based on Qwen embeddings. I revisit this project every year around this time to see what improvements I can make to make a “better” map than last year (According to my own preferences). [0/1]

One thing I learned this year is that naive text mapping (embed a corpus of reviews with a Qwen embedding model, use umap to plot, then compare where things fall) primarily compares embedded text length more than anything else. What a review says affects the position less than how long the review is.

This was a little surprising to me. For whatever reason, I didn’t think the embedding would so strongly describe the length of the embedded passage. So most of this year’s time was spent fixing that artifact.

[0]: 2025’s map: https://storage.danieltperry.me/share/steam-map/

[1]: 2026’s map (still a work in progress, both the ui and the review processing are incomplete): https://storage.danieltperry.me/share/steam-map-refresh/


Replies

PaulHoule • yesterday at 3:34 PM

Length-dependence has been a thing in vector search since bag-of-words!

https://en.wikipedia.org/wiki/Okapi_BM25

was one of very few useful results of the first decade of TREC research... And the main thing it addresses is balancing various factors which could cause a full text search engine to privilege either long or short documents.

Roxxik • yesterday at 7:09 PM

Not sure how related the cause is, but the fix seems to apply too.

In single cell genomics you often have a large correlation between the data and the length of the sequencing reads.

You can calculate the correlation between your metadata and every PC dimension and exclude the highly correlating dimensions before doing the umap. Similar can be done for other metrics. I've seen it done for patient gender for example. I could imagine looking at the correlation for publish year or genre.

Edit: fixed a misconception

tomaskafka • yesterday at 4:59 PM

I have played with similar problem in different domain, and if you could have “which users play which games” (or in general “which articles are read by which people”), you can do really good similarity search and self organizing map based on no content at all, just the metadata.

Which is of course why Facebook can say they don’t read your conversation content, while still be able to target ads precisely based on conversion metadata.

Alulim • yesterday at 7:05 PM

This is very cool! The web pages didn't run well on my phone, but that was easily fixed using Opus 5.5 and a local fork. Thanks for creating this!

Jtarii • yesterday at 2:39 PM

Interesting, a way to filter by number of reviews and review score would be nice.

➕ show 1 reply
Joel_Mckay • yesterday at 4:58 PM

In general, ranking relevant keywords in the context of the Negative reviews will likely have more hits the longer people complain.

The random fluctuations of individual experience only cancel out bias if and only if the results are truly unbiased normal distributions.

In the case of positive:negative ratio feedback scoring, it only makes sense if you ignore the entire message body of what people enjoyed.

Negative-weighted ratio ranking more accurately describes the end users experience. Any game with paying unique customer greifer total scores under 15% without game-play reliability related complaint keywords is going to be a fairly good game if the genre matches your profile preferences.

Otherwise, people will just see shit scores from the response bias of pissed off users on dodgy OS/Networks. Would recommend checking your conclusions with the average current active players in the SteamDB plugin. If users aren't engaged with the paid gameplay experience 6 months after launch, than the release likely isn't fun or broken garbage. =3

A good game, but takes some skill to play (4270 daily players 10 years later):

https://store.steampowered.com/app/219990/Grim_Dawn/

A similar genre game, but people score it higher (3650 daily players 1 year later):

https://store.steampowered.com/app/1145350/Hades_II/

A top ranked genre game, with a nostalgic series scoring bias (9651 daily players released this year):

https://store.steampowered.com/app/2536520/Diablo_II_Resurre...

plagiarist • yesterday at 3:05 PM

This is an interesting idea, do you find it has more interesting properties than doing cosine similarly by reviewers? Have you found things adjacent to games you enjoyed that were also fun to play?

➕ show 1 reply