Just this morning I had a worrying result when I googled "UV index <location>" and it confidently told me the current UV index was zero.
It was not, and at least because it was obviously wrong I clicked through to a page with the real UV index.
But had it said "The UV index is 1.7" I might have believed it, but it'd still have been wrong, potentially dangerously so.
Now to be clear, live data is precisely the sort of thing I expect it to get wrong, and I never intended to trigger let alone read an "AI summary", but it's still worrying for those less inclined to dismiss it.
They used to handle real data extremely well by using those little search widgets they would recognize what you’re asking and just show weather etc. the AI answers is just such a massive regression.
The worst example of this seems to be tide tables, which are extremely date-dependant.
It would be obviously wrong if location=out standing in a middle of a field in direct sunlight. It wouldn't be obviously wrong if location=indoors.
Proving that search hasn't stopped making us think at all. Just like we've had to do with humans since the dawn of time, we continue to have to think hard about how our intent will be interpreted.
It's almost like what we really need is someone to simply index the sites that have the actual primary sources so we can search them and find them quickly! /s
Had the same experience outside of AI recently. With the cli friendly weather reporting site wttr. It reported temperatures that were about 30°C off.
Can't trust anything anymore these days.