Audio shouldn't be a big problem for the Pi unless you're pumping it through tons of heavy filters. The Pi 5's CPU can hold its own against 2010-2015 era iMacs, and a good microSD card easily holds 40-50 MB/sec writes.
For better performance, I'd plug in a USB SSD (USB 3.0 can put through 300 MB/sec or more), or even use built in Ethernet, good for writing 100 MB/sec out to a NAS or another networked computer.
Do you have any preferred SD card brands and models? Speedy and durable cards suitable for RPi
Is there a reason you couldn't use the USB audio and MIDI gadget drivers to pass through FireWire audio interfaces as class-compliant USB devices?
You could even use a virtual MIDI interface as a convenient way to control features beyond those supported by the USB audio class, along the lines of
https://github.com/michaelforney/oscmix/wiki/Protocol