alt Hacker NewsResisting Cognitive Surrender
Comments
The tendency to over-engineer and over-build is one of the main challenges I'm seeing. When it comes for free, why not build a class hierarchy of 8 classes each with a 200 line unit test? Something I might have previously thought really hard to minimalise down to 30 lines is now bloated out to hundreds or even thousands.
Previously, pretty much everyone all agreed on YAGNI and KISS because code was so expensive to create and more expensive to maintain. Now the code generation is "free" - but at a design level, YAGNI and KISS do still matter. But they are enforced now only by sheer will power. You have to actively restrain the code from being created if you don't want it. Perfectly good code - I can't even fault it. I just know, if it doesn't need to exist yet, it's a mistake to create it earlier than needed.
I have been thinking about this for a while. I'm not a Luddite, I can see the amazing value that modern AI brings, and how useful it can be.
At the same time I can't help but wonder that this is almost too much for us to cope, the LLMs are _too_ easy to use. I think we're very good at pretending there isn't a problem here but there is definitely going to be some cognitive decline happening.
We want to pretend like reading over the output of an LLM is nearly the same as us making those changes ourselves but I think there's a lot of value in those boring and manual steps. And most people (including me) will be too lazy to not eventually offload most things to the LLMs.
In a way this is similar to how watching a lecture tends to give one an "impression" of understanding that doesn't correlate with reality. We need hands-on work and some course work to get our monkey brains engaged.
I can only imagine what effects this is going to have in education.
I think you need to pick and choose your battles.
When I'm working on my own, I scrutinize every line of backend code and frequently send the LLM back for rework. Frontend code I keep an eye on just to make sure the LLM isn't getting stupid (which it occasionally does, even at the frontier), but mostly defer. The CSS I don't even look at.
This isn't too different from having employees. How much do I review other people's work? Depends on the employee, depends on how critical the work is.
Superhuman intelligence is much easier for an LLM to achieve if humans are made dumber first.
Great post. I particularly like the 4 column pr view idea. I’ve built something similar to review speckit specs. The scaffolding to ensure that code actually works makes sense too.
There’s one thing that’s bothering me though — all of these have us consuming what LLMs produce. I learned and internalized how things worked by building systems. At least within the constraints of the abstractions I was using. This makes me think of TikTok and shorts. Everybody is just watching. Maybe that’s not charitable though. Maybe the abstraction has now changed from an “os/gui/event loop” to “llm” and the rest is just accidental complexity.
>> A more serious irony is that the automatic control system has been put in place because it can do the job better than the operator, yet the operator is being asked to monitor that it is working effectively.
Automatic control is good for standard case, and the expected errors. For weird cases, for now humans are better to detect them and press the big red button.
Maybe things will evolve into projects having a well-defined, hard boundary between agents generated code and human generated code. Where agents generated code is strictly separated from human engineered code and and can call only a strictly defined set of APIs designed by humans.
Think of platforms like Shopify. Shopify allows huge number of external users to write code customizing their web shops. But Shopify engineers do not need to review and understand the users' code, because there is a boundary which limits what the external code can do. The external code can be A Big Ball of Mud without making the whole platform a A Big Ball of Mud.
i sometimes wonder how all carpenters or music producers in the world would react if tomorrow a magic box arrives where they type-in/dictate what they want and voila... out comes a table/composition.. Will the worlds music producers and carpenters just "accept" this new magic genie? I would hope they wont. Yet here we are as software engineers...ready to surrender our craft to mere "prompts" with reckless abandon.
"Yup, looks good to me!" said the human reviewing thousands of lines of generated code everyday.
This has become common now, and I've seen it ruin large projects until nobody understands the codebase full of verbose nonsensical code and comments. It becomes impossible to further develop the project without AI. Contributors not using AI for any reason are all leaving because no one enjoys working with mindless robots (and humans) as colleagues. It's demeaning and devaluing their labor and humanity. They're not getting "left behind", they are leaving the madness of crowds behind.
> ..the possibility of runaway dynamics: once a critical threshold is crossed, small increases in adoption can trigger rapid population-level shifts toward *persistent dependence, with abrupt losses in cognitive competence*.
We can all see it happening. As this technology permeates society, the challenge to maintain control, coherence, and quality will become greater and ever more serious - for example, AI in healthcare, military, management and governance.
Isn't this the same as people who refused to use calculators when they came out?
The most difficult part of LLMs and agents seems to be - surprisingly - humans.
Specifically, humans and their dysfunctions. Their fragility. Their ego tied to stuff.
This is an excellent post/reflection on the topic of solo LLM use. The only gap I see is that it assumes that one works on something alone and not in a team. Though, that might be a different essay in itself.
Edit: Turns out that footnote 3 actually mentions that already. Nice. Good post
I have been doing this for almost a year now by limiting myself to less powerful local models via llama.cpp and pi.
Fighting over engineering is hard. But it is also one of the opportunities for us humans in the era of abundance.
I did learn a lot about the ecosystem and myself from this experience.
Cognitive surrender is intellectually convenient. As humans, its instinctive for us to chose convenience: path of least resistance. The tyranny of convenience is what drives most economic activity today, especially the gig economy[1]. Unless there are structural frameworks put in place to resist convenience (exams in schools), nature will always entice to chose the path of least resistance
[1] https://www.nytimes.com/2018/02/16/opinion/sunday/tyranny-co...
What an amazing article! It's really hard to find the balance between "ship faster, fix later because AI will help" and "efficient thoughtful engineering" these days. The world is kind of pushing everybody to work faster because "with AI you're more efficient" and it's also pushing to surrender and just accept it from time to time. Maybe that's occasional thoughtful surrender mentioned in the article, or it's just lack of energy to resist to it at that point.
The article is inspiring though, I have more motivation to work on my thinking process and my engineering setup (and vibe limit haha) to be more thoughtful, thank you.
Making a quick prototype is probably the best way to find out if an idea is worth keeping. Vibe-coding it and then throwing it away seems like a good way to go?
I am often thinking recently of the littel loop diagrams from the "The Capability Trap" paper. We need to keep a loop reinvesting into our own Human capability in anything we're doing.
At least "cognitive surrender" sounds catchier and more academic than "stupid on purpose".
I know I've cognitively surrendered when my first instinct is to ask AI to summarize the article.
I am losing more of what little focus I've had every day.
Maybe it's because I'm not working in the workforce (grad student), but I find my best workflow is to fight and argue with my agents throughout the entire process. Often times, since the agent is such a sycophant, I end up arguing with myself...I actually find this useful because it gets me to think about the other side of a problem and poke holes in it. This is not fast, but it is effective.
Excellent article.
One aspect to resistance that is oft overlooked is knowing when to fail on purpose so you can succeed on a bigger issue elsewhere.
As an example, I’m juggling multiple projects at work and home. I have a finite amount of time, and access to AI. One camp would tell me to use AI for all of it to move faster, but that increases the risk of Cognitive Surrender. The other camp would tell me to do it all by hand, but I don’t have the time available for that; I must pick and choose.
So, I choose to cognitively surrender on the SEO task for our website because that’s a set of skills I don’t want to acquire and are better handled by AI anyhow. This leaves me more time to focus on learning more about Apple’s configuration profiles and improving our MDM suite, which are valuable skills that will still pay out in an AI-first world.
You have to pick and choose where to fight, otherwise you’ll expend all your energy on losses.
> Resisting Cognitive Surrender
The economics of the whole affair unfortunately point squarely in the opposite direction.
I simply don't care. As a student I thought that the mythical production code is perfect and error-free, but in reality 99% of systems are a hot potato game sprinkled with politics where you don't want the grenade to explode in your hands.
I mean, if the management doesn't see value in decent engineering, then who am I to butt myself into the conversation?
The primary reason you shouldn't surrender cognitively is that the truth matters. That's it.
Of course, when you're working on a system of high complexity - and one that is constantly changing to boot - with all sorts of historical reasons and what not that inform the why and so forth, you can't reasonably be expected to understand everything. This is what bracketing is for. When I acknowledge "this is the way X works", it doesn't mean I have to accept the proposition "this is the right way/best way to do X" or "X must be this way". A teacher or mentor may say it is, but in a theoretical context, unless it has been justified, you cannot claim that you know that what has been said is true. However, you can accept it as a provisional working hypothesis. And practically, it may work out and perhaps experience will lend some feeling of substance to the hypothesis, but you needn't make it doctrine.
Empirical science often has the same spirit. Perhaps the model isn't a true account of reality, but it puts handles on reality so that it can be instrumentalized.
There is an occupational danger here. If we focus exclusively on the instrumental, if we reduce all of human attitudes toward the world to use and utility, we not only vulgarize ourselves and our orientation toward the world and even others, but we risk trampling the truth and the good underfoot. We risk developing an indifference to the truth, because all that will matter is some measure of effectiveness. This is what one might call the textbook[0] definition of bullshit.
[0] https://www2.csudh.edu/ccauthen/576f12/frankfurt__harry_-_on...
[flagged]
[dead]
[dead]
This is copium that is gone with the wind when the models perform up to the level of standard that the user requires. I'll give the author 6 months before they feel like its okay to just send it.
Karpathy had the same trajectory. Primeagean had the same trajectory. I think in the end everyone except for Jonathan Blow will have the same trajectory
I think there's a paradox. Is it only "cognitive surrender" if you already know how to code? Not knowing all the details seems more like the default human condition, in the sense that most people don't know exactly where their water comes from or where their waste goes exactly. "Water comes from the tap, food comes from the store".
Most of the non-technical people I see using LLMs to create software are not engaging in "cognitive surrender" as far as I can tell.
They didn't read the code before and they don't now. However, asking questions and improving understanding of the system and architecture is actually faster and easier for them now than it used to be.
They're evaluating what they can see, i.e, judging results. They build things for their own use. They test them, iterate on them to meet their needs and try to achieve their vision. This is roughly the same process they followed when they used to outsource to an agency except the loop is much tighter.
It seems to work well, mainly hurts when the problems are invisible, so security, backups, breaking migrations, perf, maintenance over time (language / runtime updates, dep updates, server updates etc). All the stuff that hurts if you have never been responsible for the uptime of something before. Not knowing that software is never "done". But junior devs trip on those exact same things.
I agree with the tail of the article; tooling and processes that make that stuff more visible would close a lot of the gaps.