logoalt Hacker News

jcmontx • yesterday at 8:15 PM • 2 replies • view on HN

I think AI papers should always include prompt / prompting history as an annex


Replies

bmenrigh • yesterday at 10:30 PM

I think everyone asking for this kind of thing doesn't really understand the scale and complexity of solving a big problem with many agents.

For any of the big things I've worked on, it isn't one "prompt", it's a goal that keeps getting updated and a bunch of back-and-forth steering of the model and sub agents, where what the human is saying to guide the model is heavily context dependent on what the model is currently doing or saying. Much of that guidance will ultimately be useless guidance on dead-end explorations. Also, the model will likely have written huge programs to solve things symbolically or simulate things numerically, and both the programs that were written, and the output of those programs are critical to understanding how the model came up with the things that led to a solution.

Now, I'd be all for publishing the full transcript, but the transcript is likely to be massive. Gigabytes of text, and reproducing the local computation (often sympy or numpy) may be hundreds or thousands of CPU-hours.

➕ show 3 replies
robrain • yesterday at 10:01 PM

If it's good enough for kids ("show your working") it's surely good enough for the bros' puffery.

➕ show 2 replies