alt Hacker NewsYou probably still better use inference on ANE (Apple Neural Engine) via CoreML rather than Metal - speed will be either similar or even faster on non-pro macbooks or iphones and power consumption significantly better. Metal or even MLX format doesn't have to be the fastest and the only way to access ANE is via CoreML.
Can use this library:
The CoreML backend is WIP in Axiom and will roll over to parakeet.cpp when it's ready, the same with CUDA. FluidAudio is a great option for those building Mac-only apps, but the goal with Axiom and Parakeet.cpp is to be very portable and embeddable into almost any app. I will write C and Swift wrappers shortly, then if it's really wanted, a Python wrapper.