Not climbing giraffeski, but have done lots of sport and taken my toddlers along (netball, trampolining) and sat them on the sidelines in their buggies. Had 5 years as a single parent and no-one seemed to mind me tagging the kids along. Also did a wall climb 6 wks after ds4 (now 15wks)was born but did have dh2 with me to help.
I guess if your thinking of outdoor climbing it could get a bit tricky leaving your little one at the bottom of a mountain!
Wall climbing was nothing - I was playing netball 3wks after having ds4! As soon as I'd stopped bleeding I was swimming too at about 3 1/2wks! It would have driven me nuts to wait any longer tbh! Am thinking of going back to trampolining but not sure my pelvic floor is ready for something so strenuous just yet!!
I used to climb (summer and winter) before I was PG, but have only skiied since. DS is now nearly 8mo. TBH, I would never take him to the crag as he'd get bored, so DH and I do a lot of hiking on trails instead which DS loves (we're in Canada at the mo, temporarily).
Oops, just read your post and seen your DD is 2!! I used to loads of kids out cragging with parents at the w/e. Generally it was a group of 3 adults - 2 climbing whilst 1 minded the sprogs. I think the big danger is stuff falling from the crag - people, rocks, gear etc etc. Not that I ever saw it happen.
Which part of the UK are you in? I know the Jacobites club in Edinburgh has a family section called the Jacobits - cute!!