A nuke plant is concrete, pumps, fuel storage and (re)processing, a huge pressure vessel, some very complex moderator machinery, and some of the most complex industrial plant control on the planet.
Even if you ramped down the safety, it still wouldn't be cheap or simple.
Are you implying that a coal plant doesn't have literally every single one of these? I have done industrial controls engineering for both and coal plants are actually quite complex. Take my word for it, they're well within spitting distance of one another, at the most basic level. The only difference is the enormous level of surety provided in a nuke plant design.
"Fuel storage and reprocessing" isn't that much of the cost and a significant proportion of that is compliance costs and extreme safety measures. The pressure vessel is likewise a small minority of the cost.
Industrial control systems are fundamentally sensors, actuators and a computer. None of those is actually that expensive. Nobody should be paying a billion dollars for a valve.
Older reactors have somewhat high operating costs because they're so old, many of them were built more than half a century ago. Newer reactors often have higher costs because of the lack of scale. If you only build one or two of something you have to amortize the development costs over that many units, mistakes that require redoing work are being made for the first time, etc. Build more of them and the unit cost goes down.