And my understanding is that we got punched cards of that size because they fit into the boxes and drawers for large-format US banknotes:

http://en.wikipedia.org/wiki/Large-sized_note

Which in turn apparently goes back to taking a common paper size and cutting it into reasonably-sized pieces:

http://ask.metafilter.com/262846/What-is-the-origin-of-the-d...

So apparently one person lost to history said, "How big should it be? I don't know. Maybe about this big?" Centuries later, somebody at IBM said, "It looks like we could fit 80 columns on that. Good enough?"