I've visited Florence and Tuscany numerous times. Sometimes after landing in Rome I've rented at the Rome airport and driven to Tuscany, othertimes I've taken the train to Florence, stayed there for a time, then rented a car in Florence and moved out to the countryside.
Both methods work. The determining factor for me would be whether you wanted to stay in Florence for any time. If you want to stay overnight you can rent in Florence when you want to leave. The car rental folks will deliver the car to your hotel, or you can pick it up at many different locations, including autoeurope near the train station. Getting out of Florence is a little daunting but if you ask directions have a map and stick to the main streets it's not too hard.
But if your destination is only the countryside and hill towns of Tuscany I'd recommend getting the car at the Rome airport and driving. It's only 3 or so hours to the Tuscan countryside, or less if you'd like to visit Orvieto (not in Tuscany but on the way) or Montalcino or Montipulciano.