Well I haven't done any studies on it but I'd assume that the number of people going to the toilet and then immediately touching the way-out door handle without washing their hands is probably higher than the number of people going to the toilet outside the bathroom and then touching the door handle on the way in...

Not to mention that bathroom door handles are pretty good conditions for germs: warm (indoors), damp (from people who haven't dried properly), lots of new germs from people who haven't washed.