The Lisp curse, now coming to your ecosystem

The real "[Common] Lisp curse" is that once you taste it, there's no going back.

Other than that, the Common Lisp ecosystem is doing well thank you very much. Many contributors to SBCL, regular contributions to the important ecosystem libraries, vivid projects (Lem), new editors (mine) and plugins (OLIVE), some companies picking Lisp and somehow supporting the ecosystem (hosting conferences, contracting implementors). https://github.com/CodyReichert/awesome-cl/ · https://github.com/azzamsa/awesome-lisp-companies/ And yes, new vibe-coded projects too. Some are useful (ICL, cl-tuition…), some reinventing the wheel (badly) went nowhere. For some, we'll see.

> I’d rather rely on proven libraries with well-defined abstractions than generate hundreds of lines of code which solve a local problem for me.

I 100% disagree with this. Supply chain attacks are already at an all-time high due to agentic hacking assistance, and that trend shows no sign of slowing. Every dependency you add is another potential attack vector, and just checking the dependency’s code doesn’t solve the issue - your dependencies also have dependencies!

The cost of rolling your own has gotten so low that it’s almost always a better security tradeoff to reimplement the thing yourself. Is that a good thing? I’m not sure.

Carmack once said something along the lines of him not reusing code a lot but rather rewriting it each time.

He is probably not alone, so this "curse" is has never restricted to Lisp.

I wrote this same concern in February: what happens when it's easier to build custom software than spend one search looking for mature libraries?

https://gavinray97.github.io/blog/llm-build-cheaper-than-sea...

> if we accept the assumption that it is easier to let an LLM generate functionality than to learn the interfaces of existing libraries and frameworks (or let the LLM integrate these libraries or frameworks)

This parenthetical seems like the weakness in the argument to me. Why have an LLM reinvent the wheel rather than let the LLM build on existing libraries?

I've been reinventing the wheel since I started programming. Back then, we had no choice--there weren't any open-source libraries. But I also enjoy reinventing the wheel because (a) the wheel can be exactly how I want it, and (b) I get to learn and understand new things.

Coding agents have actually cut both ways for me. I can reinvent the wheel more easily and with higher quality, but I can also integrate an external library much more easily than before.

The other day I needed my system to download SFTP, so I had the agent build an SFTP client. It turned out that I actually needed FTPS, so I had the agent build that too.

Another time I wanted to integrate with Git repos, but instead of trying to reinvent that, I asked the agent to integrate libgit2. Of course, I asked it to add an API that matched the semantics of my system, so from my perspective, the wheel was exactly how I wanted it.

I think what's going to happen is that there will be more vertically integrated platforms that do it all. If you think about it, it's really annoying to cobble together a system from different libraries, each of which might have different semantics.

What we want, as developers, is to have a single, well-designed, integrated platform that exposes all the functionality we need in a consistent way. That's a lot easier now that LLMs can mold code to your exact specifications.

I'm biased, of course, because that's exactly what I'm building with GridWhale.

Seems a bit hyperbolic to call it a curse. Things like these are on a spectrum for all languages and languages like C and many others are afflicted by ecosystems that are more or less community driven and leverage libraries to a greater or lesser extent. Doesn't really have to do with Lisps per se.

Actually this is not a curse but a blessing! Being able to implement any functionality without the need to resort to dependencies is a godsend.

Yes, some dependencies are great. But unfortunately the vast majority are problematic: Not properly supported, badly tested, with unhelpful maintainers, funny licenses, undocumented and, definitely, not being able to cover your requirements 100% so you have to change your requirements to meet the dependencies instead of the other way around.

So every Lisp developer creates their own time libraries and HTML DOM libraries because it's so easy? No. This is a myth.

I do think the current ecosystem of open-source libraries won't survive as it is, as AI screws up the incentives too much. But agents recreating the wheel for every project isn't practical either.

What I think we'll see happen is that AI vendors will create private ecosystems for sharing code. I.e. you subscribe to Codex and you get access to a suite of libraries that speed up your development process. Behind the scenes there'll be a mirror of the how the current ecosystem works, with agents submitting bugs and feature requests and a private swarm of agents working the most popular / high priority.

Well, if this means I will get to write code again, even indirectly with LLMs, instead of spending my whole workday merging renovate/dependabot pull requests, then I am all for it.

the irony is the repl way of working is extremely well suited for agentic workflows. rust is nice, but lisp is hiding in plain sight

LLMs stop me using open source libraries for two reasons

(1) GitHub is now full of slop. The majority of new open source libraries are now generated by AI. Many come from students who are just looking for resume garnish. Others come from devs who are more excited about being an "open source developer" than actually doing open source development

(2) With AI I can probably build something better. And if it's crap? No problem! I can just wait 6 months and rewrite it with gpt_galaxy or Claude Torah or something

At the same time I have no real incentives to contribute back. Anything I create competes immediately with a sea of slop. Any actual work I do gets identified and swallowed up by the plagiarism machine. And to be honest I only occasionally had good experiences with OSS anyway

I think we are looking at the end of open source software. Not the end of free software - in fact that may end up flourishing with LLMs and reverse engineering and the like. But the idea of programmers sharing code, for communal interest and career building, is completely dead.

So this means it's a new dawn for Common Lisp, right? :)

Application developers don't worry about binaries produced by compilers & in due time the same will be true for code produced by coding agents. I personally do not care what the code looks like as long as it actually has the features that I want & is secure enough for my limited use cases.

It seems like there are advantages as well as problems to this situation. Notably it prevents "monoculture" in software which prevent global worms. If every website implements network protocols from scratch, it would have bugs but possibly not the same bugs and vulnerabilities as it's neighbor. The extreme is every app is its own OS - back to CP/M where running a program involved inserting a disk and turning on the machine, that "interface" had it's charm.

This is rather interesting. But it misses the point somewhat.

The problem is whether or not we want to view code as a medium of communication between other humans vs code as an abstract machine that solves a problem. We know that "programs" in the general sense are fairly objective entities that can be studied objectively, which is where the famous Sussman line about this comes from.

But as a language for expressing execution semantics that a machine can then execute to produce a desirable effect on the real world (a solution to a problem), we would not expect such code to be written with other humans in mind, and as lisp is a strictly more expressive language (both for expressing the meaning of an algorithm that other humans will be able to study and understand and that a machine can execute), and that more people who program are concerned with producing a solution to a problem, the "curse of lisp" is the logical conclusion.

But it's overblown. There exist many excellent and fairly complete libraries for Lisp: Closer MOP, UIOP, ASDF, Bordeaux Threads, etc. And no surprise, these libraries are virtually ubiquitous and considered (at least by me) to be more or less extensions to the language. And these libraries are written in a fairly readable style that anyone can go and try to understand the code.

Secondly, the curse isn't really much of a curse at all. It is more likely to yield more robust programs. The assumption that code reuse through libraries increases software quality doesn't really hold up in practice: libraries that are "complete" tend to add bloat to applications; managing different versions of libraries on the same platform leads to arguments about static vs. dynamic linking, and so on and so forth. Ever had to debug .so versioning bugs on Linux? It's not fun. And yet, that is par for the course if you try to persist in the notion that "libraries and code reuse is always better than reinventing the wheel".

As a reaction to the utter failures that have occurred due to following that ideology, entire programming communities have popped up to fight it, particularly the "Handmade Hero" that became Better Software Conferences as well as "Software Should Work". And while those people aren't a huge fan of Lisp, they really do embody the Lisp mindset of "don't be afraid of building your own system in your own solution's language". And this does usually yield nicer systems to work with because programming is theory building: bringing in different ready-made libraries introduces conflicting theories that requires even more glue code to rectify and the glue code rarely foresees all the discrepancies. And so bugs appear in the integrations between them.

That's not to say that increasing "the curse of lisp" always yields higher quality. The kinds of developers that make great software with that mindset are the ones that are allergic to inconsistency at any level and so squash bugs as quickly as possible and that is what leads their software to be better. But on the practical side of programming work, you feel a lot less inhibited in trying to fix bugs when you're certain you understand the code and how the implementor decided to map the problem domain onto the range of possible solutions. This is where the issue with LLMs comes in: if programming is theory building (which I firmly believe), then what we're really after with using LLMs to generate code is a more effective technique of translating high level theory of the problem into the low level code that executes to solve the problem. As such, maybe vibe coding can work if the vibe coder diligently keeps in their mind what they're after. But usually what that means is writing notes about the desirable properties of the resultant application. And then they're just back to theory building... just at a somewhat higher level.

This is fragmentation. The advent of slop has caused many repos to stop accepting contributions, and people maintaining their own forks. I do it too, diligently maintaining repos that the original maintainers were unwilling to, or simply to add my desired features.

Since code is now cheap I think it is not the end of the world, but it does suck a bit for users, who may need to cobble together multiple forks.