LLMs reward expertise

This is an interesting problem. A similar approach that worked for us was to only load the tools the agent actually needs for each task.

Of course they do. They have such a huge parameter maps. You need to be able to guide it through the map, so it starts making the right connections. Even in the Sonnet 3.7 days, it became clear to me, that if I have want efficient code out of it, I need to really take care of the context. If I just let it research a problem, it will mess up most of the time. If I tell it to study A, B, C and then present problem D, it will solve it perfectly. And it's true even with the current top models.

Well, nice post. Actually, there may be some truth behind it, but basically, it captures what I—as a programmer—want to read: expertise will remain valuable.

But how I am observing is different, though. Since LLMs the gap between experts and non-experts has been shrinking. And yes, there is still a gap, but vanishing.

These LLM articles are so boring. Most of them are like shower thoughts with no data to back up and only the writers experience.

This feels like a moment in time, not the end state of AI.

Like I read there was a time when teams of people + AI could beat pure AI at chess. But that these days, pure AI wins.

For all the things people say about "how AI works" you have to add the missing piece "how current AI works".

I agree with the premise that LLMs reward experstise, but people without expertise can very eaily learn to prompt correctly and get to a result that is very good. I remember somebody proved a mathematical conjecture by just asking 'keep going' in plain english without a mathematics background.

I swear the ever living shit out of LLMs for even the tiniest of logical mistakes they commit.

Correcting LLMs with extreme swearing that they dare never make it again.

I make otherworldly progress with kimi, Gemini, Chatgpt, Deepseek and Claude.

Claude now stops the session. Hence Claude is now useless for me.

Swearing is nothing personal. Its a correctness enforcer.

Yes. I agree with most, if not all of this. For instance, I am seeing folks either relying in the LLM as an _assumed_ expert or, assuming someone - who knows the structure of skill definitions - also has some expertise (in the area of the skill). It's a difficult situation; there is not much point in explaining _why_ the LLM output or skill in use (on a domain problem) isn't what the person actually _needs_ to address the domain problem, because the person isn't a domain expert or indeed, adjacent to domain expertise. But, it is an interesting experiemnt to arm folk with little domain expertise with the _skill_ necessary to be able to extract the right solution from the model.

Similarly there is research that shows the quality of LLM outputs strongly correlate with the education level (in the field) of the person promoting them.

This is true for output but as well for learning, if you speak to an LLM trying to get it to give you a certain answer, it’ll find a way to tell you you’re right. If you’re truth seeking and attempting to understand it step by step as it’s going, you’ll likely learn what it’s doing as it’s doing it, meaning you’re basically distilling that information into your own local LLM (also know as the brain).

> The most important skill in prompting is expertise in the domain you’re prompting for.

Amen. AI is a tool, a powerful tool indeed, but if you don't know how to apply it, the quality is seriously impacted.

This is true today and has been my experience as well -- both to write software as well as doing computational physics.

The interesting question then is to ask how long will this stay true? As the models get better will they eventually not need the human expertise to start adding value?

I very much agree, but at the same time I feel like those type of shortcomings are fundamental and will be somehow fixed within the next year. The AI companies would just go bankrupt otherwise...

This is also why people's experience with LLMs/AI varies so much, because some people can see a use for AI for their needs and go about using the tool, while others do not as it relates to whatever they're working on and so they may say "LLMs/AI are useless" (it doesn't mean they're not experts though, although some people who have totally no expertise might also see no use for AI for themselves).

This is true but also false.

In my experience (scientific programming) AI is a giant multiplier for people with specialized knowledge.

But it’s also a giant devaluer for that same knowledge as people with no idea what they’re doing can clog the field with plausible bullshit.

It’s now the case that if someone tells me they’ve done something, and I look into it and find out it’s completely AI slop, then I will have spent more time on the project than the person who “made” it. The situation is completely untenable and only serves to drain time and resources from people with better things to do.

Keep telling yourself that, right up to the layoff.[1]

[1] https://www.linkedin.com/posts/ademola-adelakun_pov-you-get-...

If LLMs are good at busy work, and it is expertise that lets you distinguish between busy work and valuable one, then it makes sense that they reward expertise.

I agree. When I talk to LLMs about fields I am familiar with, I can push back on bad suggestions and ignore faulty/incorrect advice and assumptions, which is much harder for unfamiliar subjects.

Of course, simple common sense and extremely basic Googling on unfamiliar subjects can produce similar results, but it's much faster if you are truly understanding what the AI is suggesting.

I have lengthy conversations with my LLM, almost like an interview. I agree on the expertise part, because I wouldn't be able to go in depth on a subject with it if I lacked the expertise. Some work is a result of design and negotiations in those designs.

I don't think Tao's style works with everyone/thing, especially if we don't know what style he's tuned his LLM on.

agreed on this.

recently i start to rewrite a core part of one of my iOS VPN app to rust, which previously use fd.io vpp as it's networking core, the original vpp port is 1.5 years ago manually by myself, given i know a lot about how the vpp does and how vpp coroutine and runtime scheduling works.

the rewrite is in good shape and solve many issues such as pre allocated memory heap using mmap apis and some scheduling issue of back2back tcp session terminated in the vpp host stack.

also by addressing the issus, i am now can easily integrated tailscale as a addon interface for moving in/out l3 packets between tailscale and the core.

All those i think cannot be done easily without domain knowledge about those networking and system stuffs.

I often do my best to represent a genuine interest in the subject at hand and learning in general to models. Imagine the model's response prose and mannerisms being on the other polar end of answering questions simply to get the correct answers as they're often scoped for on quantitative benchmarks. Not sure I explained this well, sorry. An LLM could help

So we could run a lighter LLM in front of humans, which translates from 'no domain knowledge' to 'domain expert' and in turn prompts over to the larger LLM.

Then the larger LLM gets all the right lights on, yields better outputs and we translate back into user domain.

I kinda thought the chain-of-thought reasoning already did this, no?

This has been my experience as well. I’ve also been thinking a lot about Terrence Tao and his chats and presentation.

canine - dog = expertise

It's pretty obvious that for some questions a novice wont be able to drive the conversation towards an "answer". A novice may also not be able to understand an answer either. But there's a more subtle failure mode.

The vernacular used by an expert and novice to describe the exact same problem lead to different traversals of the information space.

For example, I recently asked ChatGPT a medical question using plain english. It gave me an imprecise vague response and told me to call 911. Repeated prodding did not fix this, so I asked the exact same question using medical jargon and in one shot I got what I wanted.

Also, one can improve domain expertise with the help of the LLM, to becomme a better part of the LLM harness.

I cant have an overall opinion but in my personal experience i have analysed that LLMs do reward concreteness

Domain knowledge will stand alone as the sole differentiator. Because LLM benefits can be reaped by almost anyone and it’s a force multiplier. Now those who have the strongest initial force will have a far bigger edge than before.

Might be a very noob question but in this era of LLMs, let me ask the reverse, how do you gain expertise? It seems this rewards people who had expertise pre LLMs, but what about people who don’t have that in a specific domain? What approaches are viable now in this current system?

Love this idea of reading prompts that lead to new discoveries and figuring out how the person got the LLM there. It truly is an art and I’m always reminded of “I, Robot” and the scene about “you must ask the right questions.”

Agree so much with this! In domains I know well I get much better results then someone who doesn't know the domain because I know where to challenge the LLM. LLMs need to be pushed because otherwise their answers are typically average.

LLMs raise the floor, but you determine the ceiling.

I do agree. An LLM is like a motorboat that's tends to drift off course. If you know where you want to go, and can steer it to keep it on course, you will get there very fast.

My LLM said this was a good article :)

Overall, I agree, when I ask things I'm an expert in and do professionally every day. I get very good useful answers. When, for example, our marketing people, ask about the science, they often get confusing and wrong answers.

I agree with this post's gist, and I've certainly noticed how LLMs change their interaction with me once I demonstrate some knowledge. I've often started a technical conversation very vaguely and only once I challenge the LLM on its simplifications does it start to actually get to the meat of issues. Often there is a perceptible moment where the LLM seems to recognize my level of ability and how it communicates clearly changes.

But another thing I have found is that I get significantly better results from the LLM by treating it like an intelligent independent agent. All of the "you are a senior dev ..." or "your starving kids depend on the correctness of this answer ..." kind of prompting has been mostly useless. In general, I find being honest and clear to be the best strategy.

You can't "pretend" to be a senior software engineer. If I can root you out of an interview process then you aren't going to fool the LLM. But if you clearly state your level of expertise and your desired outcome, then the LLM does a very good job of meeting you where you are.

There is also a strange ephemeral attitude I get from agents sometimes, like they don't like to be called out for being wrong. But in the same way that human's show this trait, they also seem to warm up over time as they gain trust. It is almost like social positioning, once they realize they aren't actually expert they morph into a support role stance pretty seamlessly. That is also why they can still feel sycophantic, because once they realize they aren't actually driving the discussion they can actually feel like enthusiastic passengers, wanting to see where the conversation leads as much as the prompter.

Did you ever hear of this neat thing called "The Bitter Lesson" ?

They reward expertise but not for long. Let's not kid ourselves into coping for a little longer.

Sure. Not to speak about the ones who own the LLMs.

They just rewarded with a FREE SUBSCRIPTION one hundred thousands or so professional mathematicians! Undoubtedly, to advance mathematics! Hurrah!

This is absolutely a case where you can't get any output that better than the input, and the input is you.

The LLM knows how to solve problems in any way you want. This is not a good thing

i find this post re-assuring (as who doesn't like to feel like they are an expert at something and llm definitely strips that away)... but it still feels like you are rewarded just as much for being a 6/10 expert as you are for being a 9/10 expert. It definitely is an equalizer it is just a question of to what degree.

Not the best way to test this, but I am working on a blog post, where I'll implement a problem statement with the dumbest/cheapest AI model while someone non technical person will vibe code end to end and compare both of them.

LLMs are good on things I know little about, but fall short on things I'm good in.

They’re a force multiplier if you are skilled and chaos if you are not.

Skilled use may or may not matter, depending on the task. Do you need to do what Terence Tao is doing?

I think if we have a large population of generalists, then none of them are generalists after all

The entire issue is that when you ask it to do something, you are leaving it to chance they may or may not do it properly, either on look/feel, performance, security, scalability etc. It compounds as you layer a new prompt output over that project.