I know we do things differently, but board shorts have always been a very casual lazing about short for summer time, that double as beach/pool swim wear. Watch the tv coverage of cricket matches and most of the guys in the crowd are sitting around in board shorts, a singlet (I think you call them tanks there) and a baseball cap or sombrero with beer in hand.

I can't imagine the alternatives being discussed as being granted a 'pass' for attire you can stroll around the mall/beach/bbq in all day.

Edit: Prob why they also don't have built in mesh jocks, cause you'd wear underwear underneath if not swimming.