I tried to reproduce those results, at least in terms of compression ratios, not speed.
However I would say that testing on alice29, enwiki8, text8 data is kinda cheating. Alice in Wonderland and Wikipedia are very likely part of the training data of the LLM models used there.
So I tried on HN comments from a few days ago, extracted from the text column of the public HN bigquery dataset.
Using RWKV v7 0.1B instead of RWKV v4, I get 0.962 bits per byte on alice29, and 1.156 bits per bytes on the HN comments. Still a lot better than 2.826 bits per bytes of xz level 9.
alt Hacker News
I tried to reproduce those results, at least in terms of compression ratios, not speed.
However I would say that testing on alice29, enwiki8, text8 data is kinda cheating. Alice in Wonderland and Wikipedia are very likely part of the training data of the LLM models used there.
So I tried on HN comments from a few days ago, extracted from the text column of the public HN bigquery dataset.
Using RWKV v7 0.1B instead of RWKV v4, I get 0.962 bits per byte on alice29, and 1.156 bits per bytes on the HN comments. Still a lot better than 2.826 bits per bytes of xz level 9.