alt Hacker News"The most important skill in the AI era may not be prompting. It may be learning how to solve problems using the right kind of help." [1]
Context - I have over 25+ years in software, and I have this observation - being introduced to a new codebase as a human is difficult, especially depending on the scale/size and complexity of it.
Yes, you do start to learn it as you work through it, but if the scale is truly huge, it may just not be possible to fully read and understand all the code and paths etc.
I have found systems-thinkers (I believe I am one, sometimes they are architects) to be able to kind of "see the whole picture" while not knowing all the details, to the point of being able to guess how the system/software should be behaving, even if it is not actually yet. This is a hugely valuable skill and I think takes a certain kind of brain too.
That said, I think recently I may have realized something - we rely on statistics and confidence levels in order to make statements about larger populations. If we can represent a codebase as a, perhaps stratified population of code, interfaces, docs, etc. etc. etc. we may be able to take a valid random sample, review portions of the code, and make some kind of assertions about the state of the larger system - potentially, from that.
I am trying to implement this as a side project right now to see if there is anything to it, basically, a combination of AI/LLM + stats/sampling + facilitated expert human review.
I'd be interested to know if anybody is doing anything similar.