I would agree with you but this is what it was like with search engines as well. Search engines got so good it felt like anyone could just Google anything instead of paying someone.

I think the surprising part is that, no matter how easy it gets, the majority of he population would still rather pay someone else to do it for them.

Honestly the first ten years of my career was just me being baffled I was getting paid to Google stuff. I get worried about AI in the same way but continue to be baffled that I am getting paid to just prompt AI now.

Maybe this time is different but honestly the world is so ridiculous sometimes that I honestly don’t think it will be different.

> Whoever claims to have skill in AI is very shortsighted. Whatever skill you thought you had six months ago can now be achieved with a simple six-word prompt in a standard workspace.

Apparently. Remember "prompt engineering?" That was the next big job for about 15 minutes.

It takes a lot of tokens to run a multi-agent system, though. AI may hit economic limits. Progress was achieved on hard mathematical problems at a high token cost. More than the operating budgets of some college math departments.

the bulk of the skill lies in shoring up the fundamental limitations of llms

they cannot be held accountable - therefore, you need the skill and expertise to verify outputs, or set up a verifier

they have a limit to their context window - therefore, you need the skill of knowing what context is useful and exposing that

they cost money and time to run - therefore, you need the skill of learning what parts can be handed off to code, when you can use a cheaper model, when you tradeoff accuracy vs latency

they are nondeterministic - therefore, you need the skill to set up evals, guardrails, knowing when to reach for the llm vs. code depending on the problem

they are not omniscient - therefore, you need the skill of setting up the processes and infra to supply legible context to the llms

etc etc

My kids' high school is developing an "AI Literacy Program" for all students by grade level. Since I work in tech, I was asked to participate. I attended one session and then decided it was a waste of time. Their objective seems to be to write a massive document that no one will ever actually read let alone implement. The primary focus was on prompt engineering. I told them it's 2026 and every kid in the school can write prompts just as effectively if not more so than the folks in the room. The secondary focus was on "safety" but really, for high school kids it boils down to 1) don't just copy pasta the output into your assignment and hand it in 2) verify everything the model tells you before believing it 3) don't put sensitive info into the prompt 4) don't use it to do anything illegal or stupid. Not sure this requires a committee/document/plan.

> I literally just ask a frontier model : "Pull the latest log and root cause the issue.".

While this can work, you also can't rely on it to work every single time, and still need to deal with the failures.

This is the part that gives me pause with all the discourse: you can't give up on the task you describe. You can't tell your team "something failed, but even after 20 prompts the model gives nothing good so let's forget the whole thing and go on with our lives"

So you still actually need the skills for when shit hits the fan. Perhaps it can be done in a more granular fashion with the model, and spending 30~60 min on it will do, but we're pretty far from "just utter 6 words"

If we could call someone else whenever it utterly fails it would also make it a different proposition. Same way people can try to fix their plumbing themselves, and call a pro when it's just too hard.

But who will you call to find the root cause of the issue in production ?

It seems to me as though you're describing a skill...

What would happen if you just screamed: "SAVE ME! EVERYTHING IS BROKEN! I'M STUPID MONKEY!"

I mean, sure, this might not seem like a lot of skill to you, but you at least have a clue what you are doing, and not everyone does.

I mean you could have asked it to build a log classifier instead so next time it happens you either get the solution immediately or it autos into an llm giving you its answer.

You're wasting time and tokens here.

[flagged]