I think this is only a partial answer but I've been surprised by how familiar dev leads become with the app even if they are not in the code.

They tend to stick around and they engage in the problem solving on a higher level and develop a detailed picture of how the app does and should behave. So at least that part of the expertise may come from working with an LLM to solve problems.