logoalt Hacker News

tomberttoday at 5:41 PM1 replyview on HN

Tangential, but I've found something LLMs are actually ridiculously good at is making LSP servers.

I couldn't find good TLA+ bindings for Neovim, so I got Claude to hack together an LSP server for it [1]. It works shockingly well, and it only took about an hour of arguing with Claude to do it.

I find it's not terribly good at actually writing TLA+ (with some very recent tests with Fable), so I'm not completely useless yet.

[1] https://github.com/Tombert/TLA-Language-Server-Protocol


Replies

popzxctoday at 5:55 PM

I would say that LLMs have really good understanding of LSPs, but they are not necessarily good at building them. Had I blindly followed the proposed flow, Rust Glancer wouldn't have reached a stage where it is at least remotely usable. At some point the size of the project becomes too big for LLM to fit in its context window, and with the tendence to add code rather than remove, the bloat can explode really quickly. At a pretty early stage, I did not catch a situation where LLM suggested an extremely stupid design (because I wasn't familiar with the scope enough at the moment), and it implemented a whole new parallel hierarchy of functionality that was already implemented but in a _slightly different_ form. When I realized it, it took nearly two weeks to unfuck the situation.

So all in all -- yeah, LLMs can be good _domain experts_ when you build an LSP, but a) I wouldn't trust them blindly, and b) the quality of code is still very much your responsibility.

show 3 replies