logoalt Hacker News

gouggoug • today at 5:37 AM • 2 replies • view on HN

This article is about a weekend project and not a serious product, so what I'm saying below is not criticizing the author's approach — I'm making a point about something tangential.

The post touches on my biggest pet peeves about how many software engineers currently use AI at work: they ask it to "write out a problems.md file describing the problem we want to solve" then lob the output over the fence for the rest of the engineering org to review, often times without even disclosing they used AI to generate it (it's, of course, obvious).

AI can not do the reasoning for us; thinking that because it is great at writing code that works, the same applies to documents whose primary purpose is to demonstrate critical thinking, explore trade-offs, and reason through a problem is absurd.

In fact (or IMO?), this "problems.md" document is entirely unnecessary. In many cases, simply describing the issue and providing the relevant context is enough for AI to come up with a working solution.

AI can not do critical thinking (yet), so don't ask it to (and stop sending your fellow software engineers AI-slop RFCs).


Replies

Confiks • today at 9:16 AM

> In fact (or IMO?), this "problems.md" document is entirely unnecessary. In many cases, simply describing the issue and providing the relevant context is enough for AI to come up with a working solution.

I don't mean to respond to your 'professional work' scenario, but I find it quite useful to first let it draft a PLAN.md, even when "in many cases" it would one-shot some implementation without it. I mostly prompt it with a problem statement, "write a PLAN.md", "ask me any questions in prose and numbered", "don't start implementation until my say-so". The back-and-forth questions are often pretty significant to the final result, and I can (skim) review the plan and make other adjustments.

When you ask most models to write that plan, you see an extended thinking trace similar to implementation itself, but now all the decisions are made up front. If you one-shot it an implementation, those decisions are sprinkled all over the implementation phase, while half of it has been written and some architectural choices have already been locked in. I then frequently adjust that plan to end up (IMO) with a much better implementation, or much better suited to my situation, with less shortcuts.

simonw • today at 5:47 AM

Yeah, I wouldn't do this for code that I was expecting anyone else to review.

This is actually the first time I've told an agent to write out a "problems.md" document - it was an experiment to see if stating the problem provided decent results, as opposed to working directly on a spec.

It did seem to work well though. I was outsourcing as much judgement as possible to the model, and the model did a good job at making decisions that got me to my required end result.

In this case, the AI could do the reasoning for me. Opus 5.5 and Fable 5.2 are really good!