alt Hacker NewsI reckon this type of task is crossing capabilities in a way that the LLMs (very quietly, mind) don't support.
An LLM can describe a photo fine, but it cannot tell you the exact coordinates of what it's describing. When I last tested this, you could sometimes get it to lie about the coordinates, but asking it to draw an approximate bounding box around a given feature - total fail, sometimes laughably so.
Replies
Yeah, I tried to have AI arrange some furniture-shaped rectangles on a black-and-white floor plan. Impossible. I could painstakingly guide it through writing a floodfill to find the corners of a room and parse out the dimensions listed in that room, but at that point it wasn't using its image-seeing abilities at all, and it still couldn't do things like putting a particular rectangle against the bottom left corner.
This seems like a more pertinent alignment issue than guardrails and preventing bioterror etc: does the bot even know what it can’t do? Because the eagerness with which they’ll be wrong about drawing a bounding box concerns me much more than whether they’re willing to talk me through synthesizing LSD