Well there are already offerings like githits (https://news.ycombinator.com/item?id=46105112) that sort of promise optimize bang-per-buck of inference