It's not so much a problem as the need to adjust the tricky brakes -- esp. if you're going to use the same wheels all the time, it's really a one time thing (it would be harder if you were going to use training and racing wheels that are different widths).

ETA: if the flo wheels are a good option for you, I would try them.