the guys in the crowd are sitting around in board shorts, a singlet (I think you call them tanks there) and a baseball cap or sombrero with beer in hand.

They don't have a mesh liner because they were originally for surfers (hence the name), and they were designed for fast drying, which doesn't happen as well if you've got a liner. They're really not originally designed for casual wear, and they're not designed to wear with underwear.

I thought that was just known as 'business casual' in Australia? ;-). Maybe replacing board shorts with jean shorts, or 'jorts'.