alt Hacker NewsThe precondition on the link you shared has -1 <= x && x <= 1, so 99 is way outside of that range. But even so, testing for x=1, which is supposed to be inside that range, 0.5 doesn't seem tolerably close to 0.4142.
Replies
fluorinerocket • yesterday at 6:05 PM
You're right but still, big error. I get its averaging over the range, and the floats are not uniformly distributed.
Maybe the thing to optimize the expression for is the minimize the maximum error, and not the average error. I think that's what I would care about
I have a suspicion that the accuracy number is the mean of accuracies over all valid floats in the range (or something approximating that), which is going to be weighted towards zero where the accuracy is higher, and perhaps where sqrt near 1 has some artefacts.