You are correct, but the problem was the PC only had 16 IRQs. That required using intelligent multi-port cards from Digi or Rocketport. They worked by aggregating all the ports to a single card IRQ, and managing all the hardware signals, echo.
I wrote the software for a breakout box that could handle 128 serial ports. It was an ISA backplane with an industrial 286 computer and multi-port serial cards. This was our solution for a MajorBBS system.
The BBS software would have to timeslice between all the cards handling each IRQ, then poll the card details to see which ports needed service.
GalactiComm eventually came out with their own around 1993 that could go out to 255 serial ports and did not require the 286 processor.
By the mid-90’s, Livingston PortMasters were the preferred way to aggregate serial connections, which quickly gave way to USR TotalControl.
iirc fancier (expensive) multiport ISA card had their own CPU (probably a Z80 or 8051) and a little RAM as buffer. Here an EISA card with an (unusual) Z280: https://oldcomputer.info/terminal/ap_cards/si-eisa_1.jpg