In testing for my workflows copilot significantly underperforms the SOTA agents, even when using the exact same models. It's not particularly close either.

This has lead to 2 classes of devs at my company a) AI hesitant, who for many copilot is their only interaction, having their worst fears confirmed about how bad AI is. b) AI enthusiasts who are irritated by dealing with management that don't know the difference pushing back on their asks for access to SOTA agents.

If I were the frontier labs, and wasn't billions of dollars beholden to Microsoft, I'd cut Copilot off. It poisons the well for adoption of their other systems. I don't deal with the other copilots besides the coding agent variants but I hear similar things about the business application variants.

Microsofts AI reputation is in the toilet right now, I'm not sure if its understood how bad it really is within the org.

I’ve only started using it, so maybe I’m holding it wrong, but the other day I asked the IntelliJ plugin to explained two lines of code by referencing the line numbers. It printed & explained two entirely different lines in a different part of the file. I asked again. It picked two lines somewhere else.

After using ChatGPT for the last 6 months or so, Copilot feels like a significant downgrade. On the other hand, it did easily diagnose a build failure I was having, so it’s not useless, just not as helpful.

Not even Microsoft employees like Copilot. Maybe start why not even your coworkers can use your own slop.

https://www.theverge.com/tech/865689/microsoft-claude-code-a...