Agreed. The test I run is "Can it simplify a concept I know in a matter of sentences/words, better than I can?" on a subject area that requires a lot of nuance.
And to my disappointment, the LLMs never come close. They dump paragraphs upon paragraphs. I actually want to see the LLMs surprise me. But they disappoint me for what Im specifically looking for.
Hm I gave the prompt:
> Summarize a technique for type erasure as is done in C++. Assume that your audience is familiar with all of C++'s features, but not with the technique of type erasure. The briefer the summary, the better, but don't omit any essential pieces or use vague language.
ChatGPT Plus generated a lot of text and code. Then I said:
> Now please summarize using only a few sentences.
and it replied:
> Type erasure wraps any concrete type `T` in a uniform container that stores a `void*` to the object plus a pointer to a type-specific static table of function pointers (the “vtable”). Each `T` provides its own table implementing required operations (destroy, copy/move, and any interface methods), and the container dispatches all behavior through that table. This lets unrelated types share a runtime interface without a common base class, with lifetime and semantics fully defined by the per-type table.
Not perfect, but pretty good.