logoalt Hacker News

virtualritzyesterday at 7:46 PM2 repliesview on HN

> Rust is comparatively worse, because LLMs don't make the same coding mistakes that humans do that justifies the existence of the borrow checker, [...]

That's a pile of bollocks, pardon my French. Source/proof?

And to the contrary:

I've been working on a TS codebase that calls into C++ native/wasm-compiled code for six months now. The code is mostly LLM written.

Over these last six months we had four use-after-free and two other ownership-related bugs in LLM-generated TS code.

Whereas we had zero issues of any such kind with LLM-generated Rust code that sits in another two native/wasm-compiled metacrates we use.

LLMs are not much better at ownership tracking than humans.

Especially if resource acquisition and release are far apart in code and/or somehow nested/stacked/non-straightforward.


Replies

afdbcreidyesterday at 8:34 PM

The problem with ownership tracking is that it's global. Humans are bad with global things, linters too, but LLMs are exceptionally bad at them due to the context window and (currently, at least) not knowing enough where to search. So yes I'd expect them to make the same mistakes as human and even more frequently.

show 1 reply
YuechenLiyesterday at 10:00 PM

I'm curious on what you mean by "TS use-after-free", because as you obviously know, TS is GC'd. I don't have access to your codebase of course, but it seems to me that it is an FFI/native code boundary lifetime bug in the binding between TS and C++/WASM, not part of the TS managed memory. Comparing TS to native C++ FFI and/or manually managed WASM resources interface vs Rust + Rust ownership checked resources interface is kinda comparing apples to oranges here.

And as other commenters here have said, Rust's main issue for LLMs is infectious lifetime propagation, where the borrow checker knows you violated a lifetime constraint but doesn't tell you how to actually solve it, so LLMs get error messages like:

  borrowed value does not live long enough
  cannot borrow `x` as mutable because it is also borrowed as immutable
  lifetime may not live long enough
And instead of trying to reason through the ownership graph, they just take the shortest path to get these things to go away by bypassing the borrow checker entirely, which defeats the entire point of using Rust to begin with.
show 1 reply