It might be more helpful to think of it as small changes in “genome-vector” taking place across generations of the species with the filter of it being not bad enough to cause extinction of the bespoke variation, as opposed to evolving towards more fitness as an optimisation objective. When thought of like this, one can imagine how high-dimensional the “feature”-space becomes, which leads to such intricate engineering we see in nature.
There's also typically a ton of standing genetic variation that is relatively fitness-neutral, but can then lead to more rapid or directional evolution if conditions change. For example, standing variation in silk elasticity may have been more fitness neutral when capturing other prey, but after encountering this ant species, spiders with more elastic silk were more successful and had a fitness advantage.