> They suck ass at troubleshooting systems

Given the right tools and instructions I find them to be consistently superhuman. It's unreal.

I have an automated system responding to issues occurring on oceanconnect.ca, and every single diagnosis has been correct for months. Of course I still verify, I tune solutions, I review everything. But it's right. It's not a trivial system. It has a fairly large frontend, a large-scale model processing pipeline on the backend, a fairly complex database, and a queue for getting new data into the system. Claude does a stellar job of navigating and understanding problems, even when they're not 'real' or as they initially appear to be; things like quirks in old versions of Safari, or the model pipeline fails due to data integrity issues from the providers.

I've rarely encountered cases where I need to steer a frontier LLM much when troubleshooting. They're exceptional.