Here they can't start 'proper' gymnastics until they are 6: It is run at the council leisure centre, but is never advertised, as it's always full with a waiting list - so you may want to ask/phone around.
Where we used to live there was an after-school club (but proper gymnastics with a qualified coach), and children could go from 4.
I don't know any more, but HTH