Plenty of bike shops in central to ne atl. The tri shop is Podium Multisport in NE ATL. Not sure on bike rentals though. Peachtree bikes, Intown bikes, Bicycle South are names I recall. Sure they can tell you where safer to ride, but Stone Mountain Park has bike lane, but could be crowded on weekends....different festivals and attractions. West of city is Silver Comet Trail.

https://www.atlantacycling.com/

You should be able to rent from these guys and then head up to the mountains for some great riding.

how much time do you have and how much climbing do you want to do?

If you have the time and want to climb the gaps is pretty great.