Oh @thislittleworldofmine your post outlining how this all happened, and the sexual practice (it's a kink, a perversion) he's forced on you, are just textbook male abuser.
Sadly, I assumed - before you wrote it all down - that it started after the birth of children, and that you were forced into sex that was painful. And sadly, I was right.
The breast milk perversion is just straightforwardly abusive,. A sexual practice which causes you pain, and he decides that he "needs" to have sex elsewhere?
Please, please don't blame yourself. At all. Look up the metaphor of the "boiling frog." It's how this kind of abuse starts & continues.
If you dropped a frog into already boiling water, the shock would be so great, it'd jump out again (that's us reading your thread). But if you pout a frog in cold water, and slowly, slowly, raise the temperature, it won't realise it's been boiled till it's too late.