alt Hacker NewsSo, I've been working on infinite context models (think fixed size state with a few tricks) and I think this will eventually lead to a kind of lock-in by vendor. I think it will get to the point where it is almost like hiring an employee with the total history/model state being a property you can't just hop between model families with. Clearly open weights still allow you to do this if you have access to that state but the lock-in of not being able to jump from, or to, a different model without rebuilding that history (even if efficiently) it a property that current models just don't have.
Replies
A related point is that when these things do gain object permanence and the ability to consolidate memories in a more human-like fashion, that's when the vendor lock-in effects will really show themselves.
Those memories will of course reside entirely on the vendor's servers, and there will naturally be no concept of "exporting" them or allowing the user to interact with them directly. At least not at first. Ownership of memories and context will likely end up as subjects of (far) future lawmaking. As if companies like OpenAI and Anthropic didn't already have massive incentives to establish early regulatory capture.
How are you storing the data that it can't be exported? Or that I can't use an interface or proxy that logs all my conversations locally for import?