logoalt Hacker News

ilaksh • today at 11:52 AM • 2 replies • view on HN

If you actually scroll through the transcript he links to, you will see that something that looks like it could be training is happening, but no coherent responses are coming out at any point. At least not that I saw skimming through.

That might explain why there are no benchmarks of any kind.

Replies

seanhunter • today at 7:05 PM

To pick a couple of examples:

   --- chess ---
   prompt: '<g>1700 1-0 1. e4 e5 2. '
   [raw]  repeated 8-grams 83%
   Nxd5 Nxd5 Nxd5 Nxd5 Nxd5 Nxd5 Nxd5 Nxd5 Nxd5 Nxd5 Nxd5 Nxd5 Rxd5 Rxd5 Rxd5 Rxd5. Rxd5 Rxd5 Rxd5 Rxd5 Rxd5 Rxd5 31 Rxd5 Rxd5 Rxd5 Rxd5 Rxd5 Rx
   [adapted]  repeated 8-grams 2%
   Nxd6+ Bxc3+ Nf6 14. Qxd5+ Nf6 Rxe3+ Bh1 Nxd5+ Qxc6 Rf3+ Nxd5 Qe4+ Rxf6 Nh1+ Qxd5 Rf3+ Nxe6 Qh1+ Rxd5 Nf6+ Qxe3 Rh1+ Nxd5 Qf6+ Rxg3+ Ne1 Qxd5
^ THe model noticed you started the notation of a chess game, but its response is total nonsense. After "1. e4 e5 2." you can't go Nxd6+. For all kinds of reasons. You haven't got your knight out yet. Even if you had, it couldn't get to d6. Even if it could, there's nothing there it could take. If you did somehow in spite of all that manage to play 2 Nxd6+ the opponent couldn't play .... Bxc3+ because they haven't got their bish out. Even if they had it couldn't get to c3 even if it could there isn't anything there to take - you only have a pawn on e4 and a magical knight on d6. Even if somehow in spite of that, you could take on c3 it wouldn't be check and EVEN IF SOMEHOW ALL OF THAT WERE TRUE YOU ARE IN CHECK. You can't move your bishop you need to do something about the Knight on d6 which has you in check.

All the rest of it is similarly gibberish. I'm used to model training garbage but this is in no sense AGI. It's beyond nonsense to call it that.

synctext • today at 12:02 PM

Using the term AGI and not including any performance analysis. My AI calls it: "massive marketing overreach". Somebody called this slop in the comments.

As a professor who published on continual learning I'm leaning towards agreement[1]. It lacks any substance. No relation to related work, no description of algorithm, no ablation study, just hand-waving that we're feeding some data and "Chess is not forgotten".

This "how-continual-learning-works" markdown text is not an algorithm [2].

[1] https://arxiv.org/abs/2301.12530

[2] https://github.com/volotat/mini-AGI/#how-continual-learning-...

➕ show 2 replies