Lots of good ideas above.
Please, please do enjoy taking your children and don't worry about whether they'll disrupt anything. Some churches run creches, you can phone up and find out. Others have areas to one side or at the back where small ones can crawl about on a carpeted area and where there might be some toys provided. Some churches provide "children's packs" or "bags" with storybooks, colouring pencils and pages, and finger puppets. But you can also take some things along yourself if need be - small books, small crayons and notebooks, quiet rattles and teethers, and little pots of raisins.
Again, these days there might be some people who frown but my experience is that congregations are delighted to see new people, and young families, and are quite prepared for a bit of noise if need be - they have had children themselves too!
The only thing I would caution against is letting the children run around, if they are that age, as that really does distract from the service. If your child wants to be up and investigating, there is time for that after church, and during the service, you can take them to the back and look quietly at the things at the back of church, or if need be, out in the foyer, or out in the churchyard if even being in the building becomes too difficult.
The main thing is not to worry or feel embarrassed about coming and going - vicars have seen it all before and are very glad to see people in their churches.
Another main thing: don't get worked up that you are not able to join in everything or hear everything - it might be some time before you get to hear a whole sermon, or feel able to focus on the prayers but the point is that you're there, and the "being there" is as important as necessarily hearing the message. It is teaching your children that this is important to you, and that may be the best gift you can give to God, even if you don't hear all the sermon or manage to stay in for all the verses of the hymn.
I had four children under 7 and other than four Sundays a year holiday (since my husband was the vicar) they attended church every Sunday of their lives until 14, when they were legally allowed to stay home and choose for themselves. It's not always easy but the more you do it, the easier it becomes because your children get to copying you - kneeling, singing very simple bits of the service, going to communion, crossing yourself (if that's your tradition) - and they learn how to be in church from what you do.
It all sounds a bit serious but it is so worthwhile. And come the day that they're old enough to go to Sunday-school or Junior Church, they'll be ready for it.