In similar work we're conducting at NJIT, we've found similar results. (We call it "Contextual Quality Contagion") Some of the most interesting comments here are about some of the real-world scenarios that are commonplace in industry: "mixed" quality codebases, as well as codebases with a mix of legacy code patterns with newer "better" patterns, causing the agents to confuse the conventions.

The minimal-pair design is honestly one of the strengths, as it tries to isolate the cleanliness from other factors (e.g. architecture, dependencies, tests) which is more rigorous than comparing repos. However, using LLM-generated "sloppified" code (rather than some kind of mechanical or human-guided) is a bit questionable.

I'd say the biggest critique which others have correctly mentioned here is the authors' choice to not check the full test suite. The claim of "behavioral equivalence" is only as good as the tests and coverage.

This hypothesis is compelling for two reasons: 1) it makes sense (garbage in -> garbage out as it were) since LLMs will mimic what they see in the codebase; 2) it matches what many engineers feel somewhat intuitively over the last year or two of using these models. Greenfield is almost always easier than joining a busy codebase. The mess comes in complex integration, and maintaining a system for legacy purposes, etc.