We were (still are!) in a similar situation to you(facility wise) when DD was 4 (she is now nearly 8).
She got her first LR pony at 2 and we just hacked her about the countryside on the lead to start with - walking and trotting.
By the time she was 4 we also led her off an experienced 15.1 cob so we could go further in safety.
We booked her into some lessons - on her pony - with the lady who teaches at Pony Club in her indoor school. This helped DD get the idea of steering and she could do rising trot by the time she was 3 (when she started going to little shows).
The most important thing we did, when she was 4, was to join her up to our Pony Club and she participated in their youngest lessons, did mini-camp (4yos-7yos) and by 5 she was doing little jumping courses - all on the lead rein of course.
DD is a very good rider - well above average - but she still stayed on the LR until she was 6 because we don't think children are mentally alert enough to cope with any unusal situations out hacking until that age.
In PC lessons, that were held over the Winter in an indoor school, we still had her on the LR for all activities until she was 6 because we didn't want her jerking the ponies mouth when steering over little jumping courses, learning to canter etc.
Cantering isn't easy for small people - especially if the LR pony isn't used to it. We actually bought DD an experienced First Ridden pony to teach her to canter and she has now schooled her old LR pony to canter off the lead as well because he'd never done it before. Otherwise it would have been the blind leading the blind and she'd never have sat to his exhuberance.
Ask around at school/Pony Club to see if you can find 8-12yo children to school your pony for you. They should know enough to make it work on the bit, canter, jump etc.