logoalt Hacker News

owebmasteryesterday at 8:21 PM1 replyview on HN

Context went from 8,192 tokens on GPT 4 to 1M tokens currently with zero improvement. The latest models got even worse.


Replies

joshkayesterday at 9:29 PM

Size of context is not the entire story here, it's ability to properly feed and index the context that's needed on this sort of thing. E.g. your entire slack/discord/email/github/jira/zoom meeting/coffee chat ... history is the context that you bring to the table on this sort of thing. Most of this is unindexed. Much of this will not be in the future.

> The latest models got even worse.

Which models? This is one of those things that likely has both model and domain specific aspects that impact your experience. In my experience with OpenaAI models predominantly (I previously worked there), they've improved significantly over the last 6-12 months. My experience with Claude is worse, but I haven't spent as much time getting into a mechanical sympathy there. They're still not perfect though and I have many steering docs that help avoid the biggest problems in the models I use when generating docs.

show 2 replies