That's exactly how long we were there (arrived early evening, stayed 4 nights, so 3 full days).
We spent one day just wandering - ended up walking over 10 miles! We did queue to get in the Duomo (free, but you have to wait), but if you have any intention of going up the Dome or the bell tower, you can buy a ticket and queue for much less time. We went over the river to the Piazza Michelangelo and walked back to the piazzas and churches near Ponte Vespucci - it's really lovely for eating and drinking round there. Piazza Michelangelo is the perfect place to be at sunset - the view is stunning (and it's quite a spectacle with hundreds of people there to watch it). We went to all the main squares too (Santa Croce, Republica etc). We also visited the world's (I think) oldest pharmacy, which is right near the train station. It was used in the film Hannibal, so googling that would tell you its name. Definitely worth popping in to - a stunning building.
We spent most of a day at Fiesole. 20 mins away on the no 7 bus. A lovely 'suburb' high on the hills outside the city. There are lovely views over the city; a monastary, Roman/Etruscan ruins (inc Amphitheater/ baths etc) and lovely bars/restaurants. I'd really recommend it.
We didn't go the the uffizi. We're not massive art lovers, and it struck me as something you had to really want to do, as you'd need most of a day to get the most out of it. We did go and see David at the Accademia though!
It's a really lovely city. I particularly liked the area 'over' the river (so the opposite side to the Uffizi, Duomo etc).
I'd definitely recommend lunch or cocktails (or both) at the restaurant on the top of the department store in Piazza Republica. You have the most amazing view of the Duomo, and the food (and wine) was lovely. You also have to try the T bone steak they serve everywhere - it's the dish the city is known for. Make sure you're hungry!
Also try and see the Duomo at different times of day. It's lovely against a bright blue sky, but also at dusk and when it's dark. It's really an amazing building.
We stayed slightly out of the city centre, but only 3 tram stops (5-7 mins) away. I'd really recommend the place - the owner was lovely and so helpful www.villinofiorentino.com