alt Hacker NewsIt's interesting how we seem to go through a cycle.
a) There's a gate-keeper who picks winners, and charges money. Maybe book publising, maybe academic papers, maybe music - whatever.
b) producers complain - these parasites are sucking all the money out the ecosystem, and adding no value. We need to bypass the gate-keepers. Go to market directly.
c) The "indie" scene becomes legit. Barriers to entry drop. Consumers can find creators directly. There's no middle man.
d) The indie scene bcomes popular. Everyone can publish an academc paper, or record a song, or write a book. (Of course 99% of it is rubbish, because, well, most of everything we create is rubbish)
e) along come the "guides" - folks who'll teach, or promote, or review or whatever. Most of that is of (let's be generous) "marginal quality" - assuming it's not just an outright scam (which it probably is.)
Turns out those gatekeepers were indeed providing something of value. But because they rejected almost everyone, and because they charged real money, they were disliked by creator and consumer alike. ah well...
Ultimately of course, for the 99% of us, the fun is in the creation not the result. There's no money in creating (and for the 99% never has been). Enjoy the process - if someone likes it, great. Will you make our living doing this? Almost certainly not.
You skipped a step between d) and e) - the stealth reimposition of gatekeepers under the guise of sorting through the crap. On the Internet, this took the form of "recommendation algorithms". The problem with a recommendation algorithm is that, while it is sold as "finding choices you wouldn't have known about", what it's really doing is narrowing the menu. It has to be, because of how the data flow works:
1. Obtain data on what kinds of people enjoy which books, movies, music, etc
2. Train a neural network to predict the sampled distribution. Because that distribution already contains a lot of data for the most popular works, and less for less other works, the model is trained with a bias for already popular works. Conversely, less-popular or niche works may be forgotten completely by the model.
3. Deploy the model. Because the model has a bias to predict certain already-popular works, it will recommend them more often, while the ones with little data in the set are recommended less. This changes the true distribution of the data.
4. Repeat. Take new sampled data from modified distribution and train a new model on that distribution.
As a result of applying this cycle multiple times, the tails drop out. People don't watch or read niche works, so they get forgotten. This is ostensibly fixable, you could bias the model towards less well-known works, but it's more profitable to just make people pay to boost their posts. Bonus points: people who pay to boost are also polluting the model with a bunch of negative data about their work, so they are self-selecting into a class of easy marks who don't get recommended because they keep paying to show in front of people who don't want it.