Perfect rationalization to invoke n+1 and get a tandem.
But to answer your question, it’s very difficult to answer your question. Average speed is not a metric most care about and is affected by many variables. Will CX tires roll slower than road tires? Yes. Will the affect me the precise MPH reduction that you are seeking? Dunno, could be less, could be more. You are probably in the ballpark. Try it and find out. It’s really the only way to get an accurate answer.