Very cool. I've been working on something similar. My cutoff date is 1994 as Usenet changes substantially in nature then.

May I ask how you collected the data? Usenet is not fully archived anywhere. No one has a complete set.

I am aware of:

* the NewNews disc series on CD from '91 - '93 which has a substantial text feed for the big groups during that era (most of these are on IA.org, but some are not!)

* The Dejanews archives via Google up to c. 2013. Also up on IA.org These are lacking large amounts of early posts, and are highly spotty before about 1994 (when Dejanews started).

* The Utzoo tapes over at https://archive.org/details/utzoo-wiseman-usenet-archive cover a part of '81 - '91 but only from one host. "Far parts" of Usenet over in Japan or Europe are barely covered.

* Some groups preserve per-newsgroup histories scattered all over and tending to drop offline over the years

* fj.* is well-archived but it's Japanese

The chart speaks for itself I think as to coverage: https://i.imgur.com/WCQm4od.png

I got so desperate as to go scanning zip and tar files on old ftp sites for anything with a Usenet header. That scored a few hundred thousand articles mostly from the 90s.

The big spike in the early 90s is from those CDs archives in the months where I had access. It gives an idea for how limited those Giganews/Google and other surviving archives are.

I figure less than 50% early text Usenet survives, unless someone has vast hordes of data that they are not disclosing.

As for the Utzoo archives - they're legally encumbered and some of the posters sue anyone who hosts them. (They're not available from the IA - taken down.)

You have a takedown page but the legal and copyright issues were one of the main back-burner items keeping me from making my archive live. Some of the more eccentric Usenet personalities are still alive.

Same with privacy issues. I notice you have full text search. I was reluctant to do that for privacy reasons.

I discovered very early on how to index by email but that felt quite invasive. I took that out, and built a graph of all the X-refs: which turned out to be more interesting (walking the discussion trees) and somewhat more privacy preserving in that you can explore but not search.

I was also thinking of an AI-based content warning filter, because I have a "random post" button and it is sometimes wild what that lands on.