It's hard but if you have a willing DH/flexible babysitters you can fit in some riding. Someone to share/a loan for a while would make life a lot easier, especially during the winter.
It's more doable when you have one DC. I found the second (and third) made it much tougher to find the time. Mine are on our own farm but it's a five minute drive from where we live so everyone had to be loaded into the car etc. It sounds daft but some days I was so grateful that my in-laws could feed them for me as getting there with three small DC around whatever else was going on wasn't always easy, and I often needed a hand with the children whilst sorting out the horses. No way would I have been able to look after a stabled horse twice a day and do everything else.
My horse developed sarcoids and couldn't be ridden when my eldest was still young so it was just a case of looking after her and a companion pony. In all honesty, riding once a week was all I managed before that though, but my DH could rarely be home at a time to allow me to get out and ride so our circumstances differ.
Now our children are older (9-13) , we have three ponies and they enjoy riding, one does pony club. I exercise one of them, a 13.2hh Welsh pony, during term time and that is all I have time for but keeps me happy for now.
People I know who have carried on almost uninterrupted only have 1 DC and take grandparent/nanny everywhere. If only!
Things will change a bit but if you organise help you should be able to keep going to some degree.