logoalt Hacker News

reconnectingyesterday at 7:17 PM4 repliesview on HN

Knowledge cutoff: January 2025

Latest update: May 2026

I have a very bad feeling about this lag.


Replies

SwellJoeyesterday at 8:11 PM

At least in some cases, there seems to be a move toward training on more synthetic data and strictly curated data, especially for smaller models where knowledge can't be extremely broad, because there just isn't enough room to store the world in tens or hundreds of gigabytes of model weights. So, to achieve higher quality reasoning, the training has to be focused and the data has to be very high quality and high density.

With strong tool use, it maybe doesn't even matter that the models are using older data. They can search for updated information. Though most models currently don't, without a little nudge in that direction.

Also, I believe the Qwen 3 series are all based on the same base model, with just fine-tuning/post-training to improve them on various metrics. Maybe everything in the Gemini 3 series is the same, and maybe they're concurrently training the Gemini 4 base model with updated knowledge as we speak.

show 1 reply
hoselyesterday at 7:25 PM

Can you explain what you mean?

show 2 replies
yoda7marinatedyesterday at 7:44 PM

I thought that was a choice that Google made?

verdvermyesterday at 8:39 PM

you really shouldn't have them pulling facts from their weights, they need grounding from real data sources