I mean there's no reason that the AI would need to act maliciously, thought it would be possible. I was thinking more along the lines of pre-pay for a year of server and tell the AI it has to start a business or something to pay its own bills.
If those prompt limits were reliable borders it would be very cool.
But that would mean the alignment problem was solved. When survival depends on watering down ethics, and there is a long slippery slope of ethical wells available to plum, depending on prompts for safety sounds risky.
Is that malicious? I don't think that would be considered malicious. We don't consider it immoral for starving people to steal food.
But, I am fascinated by the idea too! I just think it is a terrible idea (despite being almost certain to happen).
If those prompt limits were reliable borders it would be very cool.
But that would mean the alignment problem was solved. When survival depends on watering down ethics, and there is a long slippery slope of ethical wells available to plum, depending on prompts for safety sounds risky.
Is that malicious? I don't think that would be considered malicious. We don't consider it immoral for starving people to steal food.
But, I am fascinated by the idea too! I just think it is a terrible idea (despite being almost certain to happen).