We go to church too OP, and have a very active 4 year old DS. We found a CofE parish church which does a monthly all age service, and in general is very relaxed if DC make noise. They have a carpeted area and set out colouring and toys for the DC, but it’s in the church, so we get to have half an ear out on the service, and at least take communion. It is hard trying to keep them still and quieter in church though; we’ve relaxed a bit about not making it there every week if needs be, and have accepted getting to sit through a whole service is unlikely for a few years.
I would recommend sleep training your children. It’s not too late and it will change your life. We did it at 7 months because I was going out of my mind and DH was frankly useless and it really is a game changer.
When DS was little we, like you, both used to get up with him as it was too hard 1 on 1, and I needed company after mat leave all week. Now we alternate lie ins (not mega late, but a bit more sleep). Could you try that plus ear plugs?
Do you have any family nearby who’d take them for the occasional overnight or afternoon?
Our other coping mechanism is to not be at home too long each day. We don’t have a big house and only a small courtyard garden so we alternate beach, park, National Trust garden, soft play, extra curricular activity (get 4 year old doing swimming or Little Kickers etc), out on bikes/baby in toddler seat or trailer, country walk (short, circular), picnic or walk in the woods.
You’re in the thick of it now but it will get easier. Do the sleep training.