logoalt Hacker News

mrandish • yesterday at 10:03 PM • 1 reply • view on HN

> specifically because I'm a generalist type and I can sort of quickly cut through the space of possibilities.

I too have thought a lot about this because I've been able to get a lot of value out of lengthy, deep LLM sessions which bridge between domain exploration, research and ideation. A lot of the net value from the session isn't in the LLM's responses but in the notes I make as I'm spelunking different 'nodes in the knowledge graph' that I'm constructing, which is my own knowledge and understanding of a new, adjacent domain.

Working this way immediately felt comfortable to me because it mirrors the patterns I developed over a couple decades as a serial tech entrepreneur focused on identifying high-value customer problems which had been overlooked or insufficiently explored by others and then matching them to new technologies and processes to create a novel solution. Over the years, one pattern I found especially useful was hiring a university professor or grad student for a few hours of "Deep Q&A." My well-paid new academic friends often reported being exhausted afterward, with one describing it as "a high G-force, intensely non-linear Mr. Toad's Wild Ride" through his domain. Over years of doing this, I got pretty good at intuiting what I need to know, don't need to know and how to surface, then hone in on the boundaries and edge cases likely to yield whatever discontinuity I was intuitively sniffing out.

An entrepreneurship professor once asked to do a couple 'ride-alongs' to understand my "process" (though I've never called it that). His case study described it as "akin to an ADHD truffle pig unearthing raw, unrefined nuggets of economic opportunity". A key take-away was that nearly the entire value from the "process" was successively approximating down to the exact right "interesting question", far more than in any particular answer.

The nice thing about LLMs vs PhDs and grad students, is they're not unsettled when I hard-stop and reverse direction like an NFL running back or stall out retracing over the same seemingly barren patch until I 'pick up the signal' again and then go rocketing off in a different direction. To be clear, LLMs are not anywhere close to being a full replacement for the unique value my best human spelunking guides provide and I haven't yet seen any sign LLMs are improving in ways that will close that particular gap anytime soon (though they are improving in many other ways). I think the difference is the human domain experts have a degree of deep understanding with unbroken conceptual continuity from first principles all the way up the stack combined with a complete World Model. They can make insightful connections between concepts more than one or two degrees of separation apart (both vertically within a domain's 'abstraction stack' and horizontally across domains) in ways LLMs don't, except by widening their randomness thresholds which I've never found to be productive.

While an LLM's near-perfect recall of a vast web of statistically weighted, taxonomically-related facts along with incredible speed and infinite patience are super valuable, they're not a replacement for my domain expert human guides. For my needs, LLMs do provide around half the value of humans in this role, while being much cheaper with no vetting, introductions, appointments or breaks required. So, I'm using both in separate sessions - though an LLM able to fully track and engage both visually and verbally in a high-velocity, collaborative human brainstorm without being a 'third-wheel' would be an interesting design target for enterprising AI startup.

In short, I don't think you're wrong. There's real skill and unique value in being able to do, whatever it is I'm describing, consistently well - whether with a human or with an LLM.


Replies

epiccoleman • yesterday at 11:37 PM

Yeah, and I don't think it's just about being a generalist - obviously we know that programming "specialists" can do things with LLM-assisted coding that other people can't do.

But this:

>While an LLM's near-perfect recall of a vast web of statistically weighted, taxonomically-related facts along with incredible speed and infinite patience are super valuable, they're not a replacement for my domain expert human guides.

Really reads true to me - LLMs don't have a filter, they "know everything". So the value in the human is exactly the filter - they can kinda sieve through the mass of knowledge, weight what's important in context, and thereby extract something interesting from an LLM.

If I can get weird for a sec, there's something almost divinatory to LLMs when you're using them in this particular way - it's like a Tarot deck or I Ching or something, where you have this big collection of vague symbols (tarot) or "phrase" (i ching) that could almost mean anything. The human in that loop has to collate what subset of the symbols are interesting to the given situation. Without that filtering you just have this morass of information that doesn't necessarily get you anywhere, but when you're able to direct your attention to what's currently germane, you can get uncanny results!

(to be clear, I don't really think there's anything "magical" in any of these cases, but there's something to the metaphor!)