logoalt Hacker News

strenholmeyesterday at 11:00 AM1 replyview on HN

>>>we (and I do not exclude myself from this category) are extraordinarily bad at sufficiently imagining the failure paths that our code might take and making code work handle failure cases correctly<<<

The way I somewhat work around this with the newer coLunacyDNS code (from 2020) is by using `-DGCOV` and `gcov` to check the code coverage when running the automated SQA tests for the code. I can’t cover every single failure that could be caused by sanity tests in the C code, but I can cover pretty much all (99.53%) other code.

>>>But it's not 2005; it's 2026, and this change in compilers has been heavily advertised, discussed, complained about for well over a decade.<<<

My code compiles to the C99 standard (-std=c99 and only two syscalls not defined in POSIX) [1]. This in mind, compiler makers have a responsibility to make sure that their compilers, no matter what changes they introduce to them, conform to the C99 spec when compiling with the -std=c99 flag. [2]

This means that when I interact with people working on compilers, I bring out the C99 spec and then use that to determine whether it’s a bug in my code or a bug with the compiler. In this particular case, the C99 spec said it results in undefined behavior when “The value of the object allocated by the malloc function is used”, so that’s a bug with my code.

The thing about standards is this: A given piece of C code, if standards compliant, should, when compiled, act a given way with any compiler conformant with that standard. C developers writing C99 code shouldn’t have to look at any development or document which exists after 1999 to determine whether their code will act a given way. C compiler writers shouldn’t be telling C99 developers “well, you should know about this 2021 change to the C compiler”. They should instead say, “well, if you look at this page of the C99 spec, that behavior is undefined so we have no obligation to implement it the same way GCC does”.

Standards correct C99 code written in 2005 should behave the same way when compiled in 2026 as it did in 2005.

This discussion is like the fights guys get into when playing wargames where they argue whether a given move in the game is legal or not. When this happens, the correct thing to do is to look at the reference manual and see what that says.

>>>it's okay to seed an entropy pool with uninitialized memory in 2005 is maybe defensible<<<

Back when I made that decision, clock_gettime() was not universally implemented (it wasn’t implemented on MacOS), so my options for having some kind of entropy for the XOF should /dev/urandom have issues were very limited. I’ve since updated the code to use clock_gettime(); the Windows port will instead use the non-portable GetSystemTimeAsFileTime() (ghosts of embrace/extend/extinguish). [3]

>>>cryptographers keep complaining that we broke their code by turning their obfuscated dataflow-based if statement into an actual if statement<<<

The cryptography I use, as is typical for post-AES cryptography, makes sure that the cryptographic core doesn’t use any control flow statements, as seen in this compact representation of that code: [4]

  #define b(z) for(c=0;c<z;c++)
  uint32_t c,e[42],f[42],g=19,h
  =13,n[45],i,j,k;void m(){j=0;
  b(12)f[c+c%3*h]^=e[c+1];b(g){
  i=c*7%g;k=e[i++];k^=e[i%g]|~e
  [(i+1)%g];j=j+c;n[c]=n[c+g]=k
  >>j%32|k<<-j%32;}for(i=39;i--
  ;f[i+1]=f[i])e[i]=n[i]^n[i+1]
  ^n[i+4];b(3)e[c+h]^=f[c*h]=f[
  c*h+h];*e^=1;}
[1] The code also assumes that /dev/urandom returns a random stream of bytes, a behavior which POSIX doesn’t specify (newer POSIX finally gives us randomness with getentropy() but that spec is too new for me to assume it’s widely implemented)

[2] Until about two years ago, -std=c99 wasn’t needed; C99 code happily compiled as recently as 2022.

[3] Let me make this crystal clear: I use both /dev/urandom and looking at jitter with clock_gettime() in the entropy pool my XOF PRNG uses. Should one of those not have enough entropy, the PRNG is still as secure as the other source of entropy.

[4] I very rigorously made sure that k>>j%32|k<<-j%32 trick works to do a bit rotate while being C99 standards compliant because clang broke an earlier version of this bit rotate at some optimization values; note that j and k are uint32_t variables. Looking at the relevant parts of the standards show this trick only works when the modulo is a power of 2. The production code either uses x>>r|x<<(32-r)%32 or this:

  r = ((i * (i + 1)) / 2) % DWR_WORDSIZE;
  // Other code not shown
  if(r > 0 && r < DWR_WORDSIZE) {
                        A[i] = (x >> r) | (x << (DWR_WORDSIZE - r));
                } else {
                        A[i] = x;
                } 
The “if” isn’t a security issue because r has a predictable value which we assume the attacker already knows.

Replies

jcranmeryesterday at 2:21 PM

> The thing about standards is this: A given piece of C code, if standards compliant, should, when compiled, act a given way with any compiler conformant with that standard. C developers writing C99 code shouldn’t have to look at any development or document which exists after 1999 to determine whether their code will act a given way. C compiler writers shouldn’t be telling C99 developers “well, you should know about this 2021 change to the C compiler”. They should instead say, “well, if you look at this page of the C99 spec, that behavior is undefined so we have no obligation to implement it the same way GCC does”.

The thing about standards is this: we have the same ability to write large, bug-free specifications as we do to write large, bug-free applications--effectively none. Bugs in the specification can take years or even decades to be discovered, and then the interpretation adjudicated and fixed in a newer version of the standard, with the fossil C99 specification never being updated or given any indication that the original text was buggy. On top of that, compilers don't implement C99, they implement C99-with-compiler-extensions, and those compiler extensions' documentation range from poor to atrocious.

> Standards correct C99 code written in 2005 should behave the same way when compiled in 2026 as it did in 2005.

Standards-correct code means not hitting UB. The number of programs that exhibit UB is approximately 100%, especially in 2005 (which is about when GCC started optimizing based on C's effective type rules). The best way to figure out whether or not your code is standards-correct generally isn't to read the standard [1]. Instead, go run a suite of undefined behavior sanitizers on your code to see if your code is known to violate some of the rules. We unfortunately don't have checkers for all the known UBs (for example, effective type rules).

[1] The standard is hard to read, especially because you have to know where to track down more authoritative sources to be able to resolve interpretation issues. I'll note that you've both incorrectly identified the source of undefined behavior and incorrectly identified where to find the undefined behavior--you're citing Annex J, which is an informative section, meaning it doesn't actually mean anything as far as interpretation goes (and I'm aware of at least one entry in there which is outright incorrect).