alt Hacker NewsNot trying to be a jerk but at this point SRE is my main use case for heavy LLM stuff. It's pretty awesome at that. Deployments and CI pipelines have become a breeze. I used to have to ask DevOps for that and wait days. I have even let agents run deployments for side projects and they seem to do better work than most humans I've worked with. It's wild they will actually read all the logs and debug problems so fast. Not something we could ever keep up with.
The actual code and architecture is where it still lacking IMHO. Especially in rails... Like it will just build the least scalable features if you let it do it's thing. Ten queries for what could be one. No separation of concerns. Huge files, lots of duplicated code and then tens of thousands of units tests that just grow like a fatberg.
If your app does anything serious, if you have serious traffic... you are going to need to review each session finely (and your DB schema with each deploy). It could be that this is maybe an indictment of rails more than LLMs, I guess maybe time will tell.
Replies
Why would any of this make you a jerk ?
I guess it all depends on context, where I work ops stuff is clearly the bottleneck for a variety of reasons (technical debt that we are constantly working around, secret management for compliance reasons, etc). We self-host everything from bare metal. Some people would need to rethink the infra from the bottom up before it is "LLM ready".
That doesn't mean my job is not threatened mid/long term, in fact, thanks to LLM it is possible to rebuild that in a reasonnable amount of time I think. It's actually one of my side projects to offer this as a service. But if that doesn't work maybe I should have a plan C.
>The actual code and architecture is where it still lacking IMHO.
My experience is that the more structure and constraints you can place on what the LLM/Agents actually do, the better they perform. If you just let them run wild a la "make me an app that will make a million dollars MRR, no mistakes", you get a clusterfuck. And it gets more clusterfucky the longer it goes on.
Writing deployment manifests and CI pipelines is not really the core part of "SRE" to me: it's more related to how you run software in production, not the build/test process to get it there.
If you don't look at your code and need to debug a production issue, do you just panic chat with your agent to solve the problem?