alt Hacker News> A model doesn’t know that when the sales team talks about revenue they mean commissionable revenue, and the finance team uses something different.
Umm, the model knows the same way humans know: either by looking into slack threads/confluence pages/emails… If your model has access to those things, then it can know. And I foresee that in the near future the model can simply have an incarnation as a Slack member, and when context is not enough, they will simply ask people around (and update docs accordingly)
Replies
True, except when it’s not. The correct definition depends on the situation. For example if the if a sales director is pulling numbers to give to a finance director, then the normal sales definition does not apply. How would the LLM know that?
This underestimate tribal knowledge, wrong assumptions shared across stakeholders, same name ideas with different domains, similarities of words that can mean different things (e.g: revenue) which misleads LLM.
People have a judgment call to say “no this is impossible, when they know something”, but an llm when reads a meeting description of someone that is effusive and truly believes that can be wrong.
As someone that works on data and having seem llm full of context getting wrong assumptions, I’m still looking for the ideal thing.