It is worth having an aero road bike if you are racing or if speed/fast times are your main concern in riding.
If you need 4cm of spacers for your aero bike it is likely not providing you with the benefits that you seek.
Don’t understand this one? Do you mean spacers under the bars to make the bars higher?
Have gone down one bike size, use a longer stem and a couple spacers to get the fit I want.
Isn’t it 6 of one and half a dozen of the other choosing between bike size, fit and spacers?
Generally, no. A circular cross section exposed to the wind is very un-aerodynamic, so it is usually quite bad aerodynamically to run a small frame with the fit corrected using spacers. Some aero bikes and tri bikes have shaped spacers that likely mitigate this effect but the best fitting bike is almost always going to be the most aerodynamic.
Of course only a wind tunnel will tell for sure.