The last time I looked, all male swimming world records (distances contested in the Olympics) were held by athletes 6’0 or taller. There may be 1 5’11 guy in there, but being tall is an undeniable advantage. Don’t believe me? Lay out a 6’5 guy and a 5’5 guy on the ground side by side in the streamline position, more or less in the position they would be in after their feet leave the blocks. The 6’5 guy,by virtue of his morphology is already leading the race by >1ft.
I suspect that height is also an advantage from a hydrodynamic standpoint. A taller person would have more wetted area and thus more parasite drag, but being that humans are generally categorized as bluff shapes, it is reasonable to assume that pressure drag is dominant (especially given how effective the modern textile suits are at reducing skin friction coefficients, skin friction drag becomes an obvious secondary concern to pressure drag. A taller person would have a higher aspect ratio, or a more streamlined shape than someone of equal cross-sectional area but lesser height, so it would be fair to assume that person has lower drag (maybe in the absolute sense, but definitely in the relative sense…I.e ratio of drag to power output)