For me it had to do with multiple factors, none of which has to do with the distance. Cost, hours, amenities (gym, showers, hot tub/sauna, etc), and crowds. I chose a small franchise gym with a 3 lane 25m pool, walking distance from my residence, at $20/month. Open 5am-10pm, no lifeguard requirements, and very little use. Over two years and I never had to wait for, or share a lane. Plus, a full gym, so I could lift and use the treadmills and bake in the sauna and relax in the hot tub, etc.
For a 50 meter pool it was either the university a mile away, or swim club 3 miles away. Twice the membership fees, crowded, open swim hours were few, and always at the mercy of a life guard showing up for work. I’m not sure, but I think the swim club also required a membership to a national swimming organization similar to USAT.
Or the Y, also a mile away, also over twice the costs, 25m pool, always crowded, kids, and lifeguard. One pro is the monster state of the art gym… but not worth it to me.
My last option is the local Air Force base, since I’m retired military. It’s free. But also 8 miles away. They have a 25y indoor pool, and 50m outdoor pool. The drawback is only 3 hours of open swim in the early morning, indoors, and the outdoor pool, open June July Aug, isn’t roped off. The pool is open for anyone, mostly kids, to do whatever they want. I swam a lot there early on, but gave up when it just wasn’t worth the hassle.