It's because the way the sole slides is an integral part of bowling the ball. You can't play "properly" without a proper sliding bowling shoe.

I was a little surprised when the local bowling centres stopped bothering to make everyone use the proper shoes. I get what you're saying about playing casually, but when I last went bowling with colleagues, and won comfortably, and others were asking for tips because they just can't roll the ball properly, but won't try proper shoes, they tried to wind me up by telling me I was "cheating" by using the proper shoes that are offered for anyone who wants one - I can't win :)