You should watch "the measure of a man" episode of TNG...

What if the machine starts printing out "NOOO!! Don't turn me off, I'm alive in here"

Someone's going to go soft.

Even a chat-bot could be designed to plead for its "life".

Perhaps a new version of the Turing test is in order. Two computer programs, each pleading for their continued existence. The human judge/executioner is able to interact with both and has two kill switches, one for each. If one of them isn't turned off in 20 minutes, both will be turned off.

> Even a chat-bot could be designed to plead for its "life".

And people respond emotionally to those.

Also, see http://qntm.org/difference

Except instead of two computer programs, it will be one computer program and one human.

Chat Battle Royal.

What if the machine hires a lawyer to represent it?

America 2.0

So on the up side, I think turning it off would be the same as putting it to sleep. The question is do you ever turn it back on...