I don't know about pavel_lishin, but I care what other people think because I don't want to be an asshole. I don't want to be the guy talking loudly on his phone on a crowded bus, because that's guy's obnoxious.
The guy who's whispering to himself is fine. Weird, but at least not an asshole.
Seriously, just whisper and accept the stares. That's what I do. People lose interest in you after a while anyway.