I thought the same, but on second thought I merely had to deal on Tuesday with a lot of the mistakes Astra made on the preceding days. I wonder if this time lag of consequences explains why the sentiment is so common with these models. It probably also cautions against irrational exuberance when you first crack open a new model and it one-shots various problems, as you don't yet know what goats Astra had to sacrifice to make it so.

It's the same story every time OpenAI or Anthropic releases a new model. They are generous with compute for the first few days, and use maximum fidelity with uncompressed weights. Everything runs at its best to make a good first impression. But eventually they pare things back and the models perform a little worse.

Might be related to this announcement from Tibo on Sunday:

> We've made some improvements that improve usage on the long tail for power users of Astra when logged in with your ChatGPT account.

> No change in quality and a pure win that on the long tail can result in up to 3-4X less usage being drawn from the subscription.

https://x.com/thsottiaux/status/2096717905614524491 (https://xcancel.com/thsottiaux/status/2096717905614524491)

Disagree completely. I started using Astra from Sol the day it was released, and was a virtually imperceptable difference and made lots of mistakes and shit architecture decisions from day 1 of release.

I had nearly the exact same experience and thought I was imagining it … absolutely ripping, then it turned into Sol++ on Tuesday …

I’m working on hard things, it is very noticeable when it is hums through something and then falls over on something it should not

I can tell by analyzing my own prompts to look at when I get frustrated ;)

I wish someone ran some sort of representative benchmark suite every X days to see if this occurs.

>now it feels like Sol

It can very well be Sol, no? What stops them from using cheaper model for some requests during "rush" hours or simply use cheaper model for every Nth request.

I’m so used to seeing this on every single model release I’m starting to question if these kinds of posts are just trolling.

Alternative theory - it always seems amazing when it first comes out then the novelty wears off and we’re just meh about it. New model is a model is a model. I bought a PS5 Pro and was genuinely blown away by it at first…few weeks later I’m just like…eh it looks pretty good I guess? It’s still the same, I’m just used to it now and the wow factor along a new thing is going. Kinda like that.

Or they are just compute constrained so they have to serve a shittier version. Who knows?

I hate how opaque these companies are. It feels deceptive and evil.

Same story every time, I bet they quantized it

I had the same experience. Moving back to Sol for actual implementation.

Can you re-run some prompts that you ran on Monday and report the differences in output?

OK so it's not just me

Which plan/region are you on/in?

This shit is just vibe coder astrology lol

lol I didn't get access until Monday (I was at 0% since Friday and my reset was Sunday at 11pm), so go figure.