logoalt Hacker News

Havoc • yesterday at 11:06 AM • 1 reply • view on HN

I ended up building an MCP that pulls results from multiple places and dedups it.

The free tiers on most are generous enough for normal agent MCP usage

Also...pretty sure decodo just launched a search thingie


Replies

jimmydoe • yesterday at 6:18 PM

One thing I see on the new gen of ai search bots are they fake chrome and rarely disclose themselves in UA, and not respect robots.txt, and no rate control for smaller sites.

I’m not accusing any one engine discussed in this thread, but hearing a proxy vender like decodo providing search give me a feeling there’s non zero over lap to bad search and residential proxy crawler.

➕ show 1 reply