I like to compare swimming to xc-skiing. If you stared young you will always have an advantage since both are technical and you need to feel/know what is correct. Also, you cannot expect to be good in either swimming or xc-skiing by just working on drills. Why? The first thing you will lose when you get tired is your form/technique.

Since both are technical you always have to work on keeping or improving your technique. But if your fitness is holding you back you need to work out. The fitness will help you to keep the form; good form will not help you to be fit.

If you have the best engine and the best technique you can do what Petter Northug is doing, increase the speed in the end of the race and still keep a good form J
This is how he builds fitness
http://www.youtube.com/watch?v=Fg5fKfSuluw&feature=related

This is how he use the fitness
http://www.youtube.com/watch?v=WU1auIU_Epk

Some good video of Alistair Brownlee and swimming, he has fitness…
http://www.youtube.com/watch?v=t71f1QAmIl8