Yeah, I didn't want to put too many details as it would obscure my point. I was actually using a modified Playwright that's been designed to better act as a human. It works, in that I got fewer captchas etc than regular Playwright, but apparently not well enough.

That modified Playwright was based on Firefox, and that's what I use for my regular browsing. So that didn't help.

I can actually launch Chrome and not deal with this - but I'm not going to. In fact, what surprises me is that I can create a new Firefox profile and not get those captchas. I don't know how my using that modified Playwright as a bot got them hooked on to my Firefox profile. I suspect they had my FF profile as a "suspicious" entity even before I made the bot (I use a number of privacy/anti-cookie extensions), and the bot pushed the score over the edge.

I'll probably start fiddling with my profile - disabling extensions, until things seem to work. Thankfully, no one else in the household is impacted.

For now, I got around all this by just paying for the Kagi extract API. It's not as good as my modified Playwright had been, though. I thought of a VPN, but I suspect that won't last long (not a residential proxy, etc), and I don't want an ongoing monthly fee. I like Kagi's pay as you go option.

But as I said, it's a weird world where suddenly Amazon is coming out the "better" site/vendor. It's funny - in the old days I was pretty anti-Walmart, but a few years ago I started preferring them over Amazon (and even several local stores) for a number of reasons (better collection than local stores, not having to deal with fakes for online orders, etc).

I've not been liking the state of the web for years prior to the advent of LLMs, but this is feeling like a final straw where I accept "the web is dead". We need a fundamentally different protocol - the model of "Set up a server anyone can reach, put content out there, and then put up a lot of sophisticated technology so your server isn't hammered" is not tenable. The protocol itself needs to manage the load.