LLMs aren't people. They don't reason. They're token generators, a black box. Your analogy falls on its face with any scrutiny.

An LLM should not "generate specs", a human should. The LLM can work from the specs. It can never infer meaning from a vague prompt. If so, it will start guessing. Every human that ever did functional specification or information analysis at some point knows this. Or has learned the hard way, something with assumptions and asses ;)

> It's not surprising that an LLM would produce different specs for the same work on different runs This is what I don't understand: AI is a computer program with its own data. If we give the same input to that computer program every time, why does it produce different outputs every time? Or does the input include LLM data + our prompt + some random data that computer program picks from its Internet search?

The issue is Lllms don't learn, despite the name. A human re-implementing a spec would strive to iterate towards what they feel is a better spec. They can take in their own input and self-correct. The work of implementing the spec gives insight into pain points and strengths, even if they never actually test the spec (they 100% should, but this is to emphasize that struggle for humans is in itself iteration, even before external feedback comes in).

An LLM is isn't deterministic but also isn't iterative without an existing human. You give it the same spec 10 times and it produces 10 results that aren't far off itself but vastly different when you go into the weeds. And not different in a way of improvement. |

So what’s most important is knowing those parameters and the ranges of values, not having the final result. A human, after producing a specs, can the provide the mental model of how he created the specs. Where the inflection points are and what the range of valid results.

What has always mattered is how you decide the specs, not the specs in themselves.

> If you ask 10 different humans to produce the spec with the same information (prompt and context) they will also produce 10 unique answers

But they didn't ask humans, they asked a machine. We expect our machines to behave in predictable ways.

> If the prompt already contained answers to all the decision points that come up when writing the spec then the prompt would already be the spec itself.

This is one of the best arguments against using LLMs I've seen.

It reduces to the classic argument- at the point where you've described a problem and solution in sufficient detail to be confident in the results, you've invented a programming language.