alt Hacker NewsHere's my understanding, someone correct me if I'm wrong:
- Anthropic et al basically grabbed whatever content they could to scan and train the model.
- The vast majority of what they scanned were books they bought at wholesale in big lots. Stuff being weeded out from Goodwills or libraries and (mostly) bound for destruction anyway.
- They didn't copy the texts per-se, they studied the relationships between words to build a statistical/predictive model of natural language.
- Whether this a fair or illegal use of copyrighted text already seems like an adorably obsolete question. Regardless, the damage is basically done - they can neither prove one way or another how much any single text influenced their model and they cannot unscramble the egg without starting over.
- Rather than get stuck in court for decades splitting the hairs about copyright law, they wrote massive checks to the publishing industry and agreed to a bunch of restrictions around copyrights.
- For the foreseeable future, every independent copyright holder now has grounds to take grievances to Anthropic and get some form of compensation.
Replies
> They didn't copy the texts per-se, they studied the relationships between words to build a statistical/predictive model of natural language
This is doing a lot of heavy lifting, especially when you consider the scale
> They didn't copy the texts per-se
> they cannot unscramble the egg without starting over
I would bet they do have copies on their hard-drives (which would probably be legal for them to possess for their own use, if the originals were obtained legally), and that they could simply omit them from their next training run that they probably do at major updates anyway.
Nope. The $1.5 billion settlement had nothing to do with whether LLM training is fair use.
It was how they got the books for training: they torrented collections of pirated ebooks. There are Slack messages from executives describing the process as "blatant violation of copyright." They were stealing books and they knew it.
They didn't start buying books legally and scanning them until much later.
The vast majority of what they scanned were books they bought at wholesale in big lots. Stuff being weeded out from Goodwills or libraries and (mostly) bound for destruction anyway.
No, the lawsuit was because they illegally acquired the books (e.g. downloading them from piracy websites)
These days they are sticking to more legal means like the one you described
> They didn't copy the texts per-se, they studied the relationships between words to build a statistical/predictive model of natural language.
I understand exactly what you mean by this, and you are wrong. If they did not copy them, it would not be possible to extricate copyrighted works out of the model like Harry Potter. You can argue against the copyright system, but don't pretend that what they've done is essentially an advanced form of copying. The data is in the weights.
The last court case, mentioned in the article, found Anthropic guilty of pirating eBooks. That's where the penalty came from.
The court ruled that it was fair to train on books, but they had to acquire them legally. That's why these companies buy physical books and scan them now.
The Stanford professors in this article opted out of that last case and are trying to get higher payouts.