What's "taken off"? Not everything needs to be Google-scale to succeed - the world would be better with more, smaller companies.
Kagi is stuck at 75000 subscribers, when they should have at least a million. Google scale is unrealistic, but they haven't taken off.
Internet search isn’t one of those. Breadth, depth, and quality of search results (ie what you want is the first result) matter. That’s only reasonably attainable at scale. Not to mention it’d be a PITA to optimize for a dozen competing crawl standards.