You can't accidentally arrive at a building, that's by definition true for all architecture, it doesn't fall out of the sky. They also didn't design it to be ugly, that's your value judgement, not a property of the architecture.

What brutalists meant when they talked about honesty or the ethic of design is that the building should reflect on the outside, its construction, its social purpose, and its function. Obviously that's a visual choice, but not an aesthetic one.

And the fact that so many people, including you reach for the description of it as 'ugly' proves that point. They weren't trying to appeal to some established notion of taste, or some aesthetic school that prioritizes the appearance of a thing over its function but the opposite.