Okay, I'm going to make an important distinction here. YOU agreeing to have sex even if you're not especially wild for it is one thing. HIM demanding that you do so is quite another thing. From what you've said, it sounds like he is doing the first thing above, requesting that you consider agreeing to do this, NOT demanding that you do it.
Also, it sounds like the two of you are caught up in a pretty common dynamic. Many women don't feel like having sex unless they feel close with their partner first, whereas many men feel close to their partner BY having sex with them. Neither way is right or wrong but I think it's good to realize that you differ in that way.
And, when you have little kids, jobs and so on, the early, lighter and more exciting days of marriage are over and it's also common to just feel burdened, tired and bored with life and to feel like getting through the day is about all you're up for.
All of the above, including you sleeping in separate bedrooms, is also a very common predictor of divorce.
So I'd think very carefully about which way you want your marriage to go, when deciding how to proceed.
What I do with my husband is usually give him sex when he wants it. I know I don't have to and of course I make an exception if I want to but I do prioritize his sexual desires. It also has a way of causing him to then be more in tune with me.
I also sleep in the bed with him every night. Those are two special, unique things that exist in a marriage, you could say honored positions that only you have with him and he with you. But if you refuse to take them and clearly don't want to take them, you are sending a big message to your partner and also leaving a big gap between you in your marriage.
I would start sleeping in the bed with him and, IF you can do it without strong negative feelings, I would have sex with him when asked. AND I would also figure out a way for regular date nights. You might also try marriage counseling. So many couples don't get it until the distance is allowed to go on for too long and the marriage breaks, usually by someone else stepping into the gap the two of you left and whichever partner is targeted feeling so alone that they go with it, when they wouldn't have in a healthier marriage. Then all that you didn't do this, well you didn't to that bickering seems quite trivial.
That's my advice as an older wife but of course it's up to you. Best wishes.