OP I don't think there is any point at looking at the church thing "in isolation". This isn't an intellectual exercise, you can't look at that issue out of the wider context.
Your relationship sounds like it is really struggling - you struggle to hear one another and communicate in a way that doesn't breed anger/resentment. Your DH sounds dismissive, difficult and defensive. You sound overburdened at resentful at the unfairness of how things have panned out.
At the same time you are dealing with the emotional and attachment difficulties inherent in long separations and then the subsequent reunions.
I think the church thing is just an illustration of what is a deeper problem.
We are a clergy family. Going to communion on Sunday is the cornerstone of my DH's life and his spiritual discipline. It's not a hobby, often he can't be arsed, but he shows up anyway. I respect him for it, and he helps me to keep to that discipline too as I can sail along in his wake. If it were down to me I wouldn't bother so much and my life would be the poorer for it.
We almost always go to church together as a family and then out for lunch afterwards. But if from time to time I say - "do you know what, I'm knackered, the kids are knackered, we need a lazy Sunday, I'm skipping it this week", then he will get up earlier, go to the 8 o'clock and be back home at 9.
Whether you are being U in asking him not to go every week depends entirely on your relationship and what he is like generally. I feel I would be U to tell my DH he couldn't go, as he is always prepared to try and bend in other ways and listens to me. But given what your DH is like I am not sure that you are BU. Or to put it another way, you wouldn't mind him going if things were otherwise good?
Other thoughts having read this thread - I do find it interesting how absolutely horrified people are that someone could say that God is at the centre of their lives, more than their children even. I love my children deeply. But they are not the be all and end all, nor a project, nor the meaning of life. In that sense, God is more important to me than them.
And - "Family Time" (capital letters) : the new Holy of Holies. Woe betide any heretic who doesn't buy (and it often is buy) into Family Time. I'm all for having a nice time as a family. But Family Time? Shudder.