Thanks very much for your advice -- this is just an update for the purposes of the archives if anyone's ever searching...
We booked into the Art'Otel on the Buda side (not to be confused with the Art Hotel on the Pest side) which had very friendly staff. They provided a travelcot but it was one of those big ones so not terribly practical for a 3-month old unless you had extremely long arms or could levitate him, so I was glad we'd brought our own.
I'd strongly suggest a carrier or sling for a baby this age if you're planning to do much sightseeing as there are a LOT of steps on the Buda side and a stroller could become a bit of a handful.
Szecheny baths were great -- we didn't take DS into the water but he had a great time people watching with one of us while the other one was swimming.
Many taxis are scary -- one journey felt like the Knight Bus trip from the third Harry Potter movie. We took our car seat with us.
DS actually did very well walking around -- he'd drop off in the carrier. And I finally mastered the art of breastfeeding discretly on the move so from his point of view he was happy with everything on a plate.
I wouldn't have wanted to go much later in the year as it was already starting to get a bit hot.
Hungarian old ladies feel free to come up to you in the street ans criticise the way your child is dressed (fortunately I'd been warned about that in advance).