Upper east side is asphalt green which I think is $35 for a day pass. Pretty decent facilities and depending on the schedule sometimes a 50m pool.
Upper west side there is the JCC which if you go into membership office they would probably give you a 3 day pass if you pretend like you live here and want to join. 25m pool, only 5(?) lanes and generally pretty slow but if you don't go at a peak time you will be fine. (I might even have a guest pass laying around somewhere if you want it)
Others will probably have some more detailed suggestions, I think there are some masters practices that you can drop-in at Baruch.
No idea if this is a private club but it is much closer to you. Always been curious about it.
http://www.clubcorp.com/...able-Center/Swimming Edit: Also forgot about the Equinox on Columbus and 67th- pretty nice saltwater pool. Might be able to buy a day pass but could also get a free day if you act like you want to join.