It wasn't Canonical's limitation. Mozilla can confirm: the barriers to enter the phone manufacturing market are immense. Samsung, Ericsson, Sony, LG, etc, all had solid control of the deals with the network operators worldwide. It's a highly regulated market, you can't get in without Verizon, att, Vodafone etc to let you in. Canonical, Open Moko, Mozilla... Didn't have the strength to push through. Google and Apple did.
Worst case we can buy phones and physically replace their Application Processors with liberated ones.
It's operators that have grips on manufacturers. Phone OEMs don't make any without the sales networks and financing structures of cellular carriers.
This is to extent that some carriers still insist that they design phones and phone manufacturers are just factories. People need to understand that power dynamic before blaming those "mere factories" for not making Linux phones.
There has to be like an SVP of SoandSo Wireless pushing like three pillar approach of iOS, Android and Linux. Otherwise the pitch decks for any alternative phones just go straight to trash(or into a motorized shredder if it is still physical).