logoalt Hacker News

adiusyesterday at 6:42 AM5 repliesview on HN

Author here: I absolutely do not understand this mindset. It has almost 20K unit tests by now and hundreds of full end-to-end tests of complicated scripts to ensure it works and matches the output of Wolframscript. Why does it matter that I was using Claude to help me implement it?


Replies

mycallyesterday at 2:40 PM

https://github.com/ad-si/Woxi/tree/main/tests

Spot checking, I don't see any issues.

e.g. https://github.com/ad-si/Woxi/blob/main/tests/list_tests.rs

  #[test]
  fn first() {
    assert_eq!(interpret("First[{1, 2, 3}]").unwrap(), "1");
    assert_eq!(interpret("First[{a, b, c}]").unwrap(), "a");
    assert_eq!(interpret("First[{True, False, False}]").unwrap(), "True");
  }
theplumberyesterday at 6:58 AM

Are they 20k unit tests or sloppy tests? Would 100k unit tests make it better?

bonziniyesterday at 7:59 AM

My issue is that Mathematica is essentially a term rewriting system. Reimplementing everything in Rust seems to go against the idea. The derivative computation is 400 lines of Rust and could be 20 lines of Mathematica code.

show 2 replies
IshKebabyesterday at 8:24 AM

AI has a tendency of "just make it pass!" (which to be fair you also sometimes see from junior human devs - maybe where it learnt from!). Remember that C compiler which didn't even do basic error checking because that wasn't checked by the test suite?

A very young project written by AI means you haven't reviewed the code and nobody has used it in anger. It might work perfectly, but my experience of AI so far says that it won't.

neofrogyesterday at 7:29 AM

It could very well be 20K slop tests