It depends on the swimskin and also how long the race is.
I’ve noticed the more expensive B70 and Profile Design swimsins (speedsuits? are bonded together and constructed of fabrics that absorb no water and have the same hydrophilic treatment as wetsuits, but without the rubber. Add in the compression to keep everything tight, and you have a garment you can swim fast in, but might not want to bike or run in. (Though it’s been done by somebody who forgot their race suit.)
In a longer swim, like 70.3 and IM length swims, this will make a difference. Plus the fabric and pad in your race suit may, over time, absorb some water and slow you down.
I’m not a rocket scientist, but I am a neoprene and speedsuit geek from my days at QR, Zoot, and Profile.
KS