logoalt Hacker News

pu_pe • today at 12:12 PM • 6 replies • view on HN

> It’s got some very expensive practices, where near-free practices would produce the same or better results.

I think tokens are near-free already, even for those of us outside Anthropic.

More importantly, your philosophy comes from a time where thinking had to be front-loaded: you try to predict everything that can go wrong, then design deterministic tests that can catch those things and a decision tree to figure out what to do in each case. This approach was the only viable way until now, but maybe that will change. Having the equivalent of a senior engineer thoroughly review each error with full context of what the application and feature is trying to achieve could perhaps uncover different insights, or cover edge cases more smoothly.


Replies

Grimburger • today at 1:28 PM

> tokens are near-free already

How long do you think that will last?

Am over 11 billion tokens this month on a subscription. In API usage its close to $5000 and there's still 13 days left in the month.

Eventually the music in musical chairs stops. Already many anecdotal reports of the always opaque limits being lowered across the board. On a gemini pro account for personal use I can use a weeks worth of Argon in about 20 minutes on agy in the last month.

ChatGPT Pro I can use the weekly limit in under 2 days, this is with significant context clearing and handoffs.

The cost to provide frontier models isn't cheap and it's entirely subsidised by a winner-takes-all mentality from investors which in all likelihood won't come about.

➕ show 1 reply
skohan • today at 1:03 PM

It's great to have that option available, but it's incredibly inefficient to just throw GPU hours at a problem that can be solved deterministically.

➕ show 1 reply
castis • today at 1:09 PM

> Having the equivalent of a senior engineer thoroughly review each error with full context of what the application and feature is trying to achieve could perhaps uncover different insights, or cover edge cases more smoothly.

No one is debating this. The debate is that an agent performing validation work via its own chrome plugin and internal reasoning will always cost more than performing that same validation work with deterministic tools.

➕ show 2 replies
jgalt212 • today at 12:46 PM

As far as timelines go both property based tests and fuzzers predate this token maxing strategy.

➕ show 1 reply
NewJazz • today at 4:04 PM

Great, I'll mail you a quarter and you give me your Anthropic API key. Deal?

skydhash • today at 12:27 PM

> you try to predict everything that can go wrong, then design deterministic tests that can catch those things and a decision tree to figure out what to do in each case

No real software engineer think that way, as things that can go wrong are near infinite. Instead you design the right thing and write tests that verify that it’s indeed the right thing and add some more that are the wrong thing, especially at the edge of definition. Then you rely on the dev to not make very obvious errors like coding something that is not in the requirements or is completely opposite.