alt Hacker NewsUber president says AI spending is getting 'harder to justify'
Comments
Now feels like a very good time to be a small team of experienced developers who can largely work on stuff by themselves and not a corporation of hundreds of developers of varying abilities all now trying to show how much code they can generate and how many tokens they can burn.
Anthropic's annualized run rate is >$40b according to outside reporting. AWS hit that by Q4 2019. There were still debates on public cloud vs on prem at that time, but by late 2019 public cloud had facilitated the creation or adoption of entire categories of software within SaaS and PaaS, not to mention consumer internet businesses like Uber and Airbnb. The net impact of AI coding tools is far more ambiguous in comparison.
The profitability comparison is fraught but worth noting that by then AWS was already extremely profitable.
What has been the end result of all the tokens companies are burning?
Where does it show up in quarterly results?
I can’t see how it’s sustainable just based on “this feels more productive”
Didn't google say that AI had increased their company's productivity by 10%? if that's the case, then how can they justify spending 50% to 100% of wages on it?
This doesn't account for the fact that review is still a bottleneck, engineers understand much less of the code they're shipping and there'll likely be tech debt they'll have to unwind in the future.
There's also the issue that in any large-ish org, code production is hardly ever the bottleneck.
I am not sure how uber is operating internally around the use of tokens but if they actually shipped features faster than before then it is still a win. if they learn that users don't want these features or want a different version of it; they have learned this new knowledge faster than they would have if they manually coded those features, which means in principle you should be able to iterate faster. but this will collide with creative ceiling that humans exhibit in a span of time and on top of that uber is prioritizing spending money on tokens over humans which seems like a mistake. you need humans for creativity.
I still get picked up by an Uber the same way. As an end user, nothing has changed for me.
So I wonder what the heck were all those billions of AI tokens burnt on that they extinguished it in just 4 months into the year?
You don't need justification to spend other people's money!
Nobody's going to jail.
I actually think Chinese models have already popped the bubble, we just dont see it yet. the only way to justify AI IPO market cap is basically if they get to hold most sw industry code hostage and then token-flate budgets to collect AI tax. short of that AI expense would very quickly mean revert to model + some margin. this means the moat for AI 'trillion club' is gone. In fact AI virtually guarantees that there is no execution-moat left anywhere, definitely not in code or that engineer with knowledge about that obscure mechanism. without the moat most of the sw ecossytem's margins would shrink (as they should).
Ironically enough the only moat left would be what you can buy from Washington.
Goodhart's law strikes again. Stop giving your engineers token-burning quotas or they'll burn tokens.
I do find it to be true that with coding agents the famous quote from Jurassic Park goes through my head multiple time a day
"our scientists were so preoccupied with whether they could, they never stopped to ask if they should.
I've now come to the realization that if I'm having an llm work constantly all day writing code for me i'm probably doing something wrong as I'm no longer focusing on the core issue itself.
I may be in a minority here in that I write code to augment my self and not to ship to others so I can tell very quickly if I'm just gold platting something or if i'm actually delivering real value to my trading or risk management.
Affordable inference will be around longer if more Big tech companies cap their AI sending.
It feels like maybe the wheels are starting to fall off the AI hype train. I expect complete collapse once people start figuring out that the numbers on all this don’t make sense. I’m looking for investment portfolios that will weather that storm. If you are reading this and have a similar curiosity, this is a great place to start.
https://portfoliocharts.com/2021/12/16/three-secret-ingredie...
hot take: token spend can be used a honey pot, especially when compared to what you deliver. spend accordingly!
My concern here is that they'll mix two things:
1) workforce reduction
2) AI spend (reduce tokenmaxing)
They'll expect fewer people to do more with even less, while "more" is continuously increasing.
When I say "more", I mean that the deluge that engineering teams deal with comes from two sources:
1) the business side of companies - marketing, sales, solutions teams, etc.
2) outside actors, mainly security threats
The first source can now move to generate work for engineering faster than ever. They expect the nerds to do what they're told and get the features out now. The more features, the better the product, right? The saving grace here is that they're bound by the same management concerns that engineering has. There's only so much money that they themselves can throw at generating more work for engineering teams, and that might also come under scrutiny from management, so that acts as a brake.
The second source has no such brake, especially not with security threats. Either there's good money to be made by holding company data hostage, or there's an endless supply of resources (read: nation-state resources) dedicated to the effort to attack the company's digital assets. And of course, they're using AI to enable this, just without the "but what about the shareholders!?" handwringing.
If you aren't very, very careful with your token cutting, you're going to put yourself at a disadvantage against that second group.
I never took tokenmaxxing to be about improving productivity directly; mundane feature work that comes out of it is just a side effect. I always saw it as a race between these big tech companies to get a generational advantage by being the one to discover the way of the future, with respect to harnessing AI to actually and truly automate software development.
EDIT: whoa, I used "way of the future" as a reference to Howard Hughes in "The Aviator", not this Way of the Future religious organization thing I just stumbled on; no intended reference there.