The point of this was always to explore what is possible with AI as quickly as possible. Obviously, there is going to be a lot of waste, but the 5-10% of employees who are truly thinking about it and discovering novel applications are what you are truly after. Because right now, you effectively have a giant, as of yet poorly explored space of potential uses.

Anyone who can find the actually valuable portions of the space early has a potentially huge competitive advantage. Even if the result of the experiment is the negative that AI is actually mostly not that useful, that is still extremely useful information in a time of great uncertainty regarding outcomes.

The bottom line is that this approach may be expensive, but if you have the money to burn, it's far from the worst strategy if you are trying to position yourself correctly for the future.

I think unfortunately it's not about what seems obvious, or even what seems more likely, but about what seems retrospectively justifiable regardless of outcome.

The incentive structure of this type of decision is 'absolutely under no circumstances existentially mess up'. Ostensibly with respect to the organisation, but in actual reality much more so with respect to the individual(s) involved in the decision.

If everyone else is doing something that kind of obviously makes no sense, and you decide to break from the crowd by instead doing what does make sense, then there's a pretty solid chance of gaining a temporary edge while reality resolves the truth. But those gains probably won't matter all that much for the organisation, or indeed your position within it. It's a solid chance of an unimportant gain.

However on the other hand, there's a tail risk that something very unexpected happens and the thing everyone's doing that makes no sense actually turns out to make sense - sometimes even for entirely unpredictable incidental reasons - and then, well, you're in trouble. Not necessarily 'you' the organisation.. they'll likely be able to catch up and it won't matter that much. But for 'you' personally, the decision maker, it's very much not good.

As a bonus, in the much more likely scenario that the thing that makes no sense turns out to indeed make no sense, you're in the same boat as everyone else, there's no relative loss, and most importantly you don't stick out as someone who did something as risky as to go against the prevailing, albeit pretty clearly nonsensical, sentiment.

So basically, game theory tells you pretty quickly to just go with the thing that makes no sense if you're optimising for some (weighted) cross of what's best for the organisation and yourself as the decision maker.

Someday maybe Goodhart's Law will be intuitive to people making decisions like this, but not any time soon I guess

> It's amazing that it took months to figure this out

We aren’t there yet, so far it is just a COO questioning the investment

The inability of leaders to understand Goodhart’s Law is always a sight to behold. They see a number go up and pat themselves on the back for how well their employees are making it go up without ever wondering if the thing they care about is happening.

You say "amazing that it took months to figure this out" as if the answer to the question is obvious.

But it's not. Some FAANGs are doing amazing things with unlimited tokens. Other companies have no clue what to do with tokens, they've just told their engineers to max them.

It really depends on how you're using the tokens. If you're just using them for Codex and Claude Code - yeah, tokenmaxxing is incredibly dumb.