Also, a typical consumer camera is going to struggle (even with the best analysis!) to have visual accuracy within even a few degrees of true (due to everything from high compression ratios in post processing to shitty optical properties in the actual camera) and that makes a probability cone that is pretty large.
Still better than nothing, though!
building a WCS model (pixel XY <-> ray vectors) of a camera is a solved science. Of course you can never expect it to be rectangular.