alt Hacker NewsIs this at all possible to run locally on a MacBook pro m5 (48gb ram)? What kind of performance could I expect? Or would you run this somewhere in the cloud? What HW / which provider would you choose (single user for exploration only)
Replies
mattstir • today at 3:16 AM
Laya is only ~400M parameters and runs comfortably in 1-2 GB of RAM. In my local testing, it's very fast even using CPU. I'd say give it a try!
Easily. It took around 4GB VRAM on my nvidia card so a 48GB unified mem should have no issue. < 1s inference