Is reserving 6 bits really that bad?

You can use different encodings based on context, just like with a random blob of bytes.