Thanks for all the replies. Real good info from all. Here’s a follow up:
I rode the 56cm base Allez last week. It felt great! Very comfortable, smooth, solid ride. I really loved how this bike felt when I rode it. My only issue was that if seemed like I got up to top gear too easily. I knew I’d be wanting at least another gear on long flats and downhills. I still made a low but fair offer, but he wouldn’t move much on price, so I walked.
Instead of looking at the 54 (2x10), which was a 2012, I expanded my search area. I found a 2015 Allez Sport (2x9 Sora with carbon fork) listed for the same price ($500) as the 2013 base Allez I had checked out. It was posted that day. It was a bit of a drive, but worth the trip. Bike looked brand new and test ride felt great, so I bought it. Happy with my “new” used bike. So, in the end I found a bike that was correct size, newer, with groupset between the two bikes I was looking at.
Regarding my original question on fit v. components, I think one could go either way to a degree. Yes, of course fit is critical, but if I had found a really nice higher end bike for a really great price that was a size smaller, I would be ok buying it and making it fit.
I have that era Sora on my roadbike. Really have no complaints other than that the (completely unnecessary) gear indicator needles sometimes rattled/buzzed on chipseal surfaces. Hasn’t been a problem since I switched to larger, more supple tires and backed the pressures down a bit, though.
I like that 9 speed cassettes are cheap; for $50, you can have a couple alternate gearing ranges in your toolbox for whatever ride you might encounter. It’s flat as hell around here, I usually ride a 14-25. But I also have a 12-26 or 11 - 28 I can put on for a ride with more elevation changes.