logoalt Hacker News

godelski • today at 4:37 PM • 1 reply • view on HN

As an example of this, you can have complex code made of entirely simple and easy to understand functions. But the complexity appears in the combination. That complexity can be far higher than were you to have more complex functions in the program. Which this is a common mistake people make that lead to tech debt. A lot of people try to simplify code by making it "understandable" without setting a floor to who needs to understand it and with how much time. If you don't, that floor gets lower each year.

If you can't think of an example of the first case, most machine learning models suffice. Take a simple feed forward ReLU model. It's trivial to understand ReLU. It's nearly trivial to understand the individual neuron. Yet I, as someone who wrote their PhD thesis on neural architectures, would call bullshit on anyone claiming they "really" understand what's going on in a deep network.


Replies

itsalwaysgood • today at 4:41 PM

For sure. Binary is another example: it's just 1's and 0's, a bunch of on and off states that only make sense when you consider the entire sequence.

And in physics, everything is reduced down to 'simple' forces of nature. A bunch of interactions and relationships that add up, and if you can keep up and perceive it all, you can sometimes stay ahead of all those interactions and imagine how it works through intelligence.

Eventually, you reach a state where you understand enough about the environment to translate ideas from one mind to another through language and your understanding of the world.

And so some will have deeper understandings, and in different areas, and all manner of conflict and resolution arises as people with different understandings try and understand one another.

There's no end in either direction you look, but zooming in always reveal some simple interactions at work.

➕ show 1 reply