But it isn't just one issue. It's many, many worring issues.
Your husband is only happy if you're doing what he wants. You say that a lot of the time, that's fine, as you want the same things. But that isn't the point. You should be able to do anything YOU want, without his judgement. And I'll put money you HAVE changed your behaviour to please him, or for an easy life. And you've just convinced yourself you like the same things.
In a healthy, loving relationship, both people love the other for who they are, and they respect their differences. Noone is in charge. Noone controls the other with moods or silences or harsh words or withdrawing love.
Your husband wants to sleep with other women. He's trying to involve you, despite knowing you don't fancy women (and presumably despite knowing you don't want either of you to have sex with other people). Would he have sex with a man if you asked him to? I suspect not - so why does he think you should do this? It's because to him your purpose is to make his life easier, better, to do his bidding and make him happy. He doesn't see his job being to make you happy.
He is controlling. He's abusive. He's made it clear he wants to sleep with other women (the next threat will probably be, well he'll leave you out of it then, but it will be your fault if she has affairs - or he's already having an affair and this is part of his cover up). He'll find a way of it being your fault that he's unfaithful.
And it doesn't surprise me at all that his main strength with your daughter is 'getting her to do things she doesn't want to do'. He is a master manipluator. He enjoys power. He likes to control the people around him. He is not a good man for your daughter to grow up around.
What advice would you give your daughter if she was in your position?
Finally, you don't need his agreement to get divorced. Most divorces, I would imagine, are not by mutual agreement. He doesn't need to agree to it or be happy with it.