I am vegetarian by moral choice. I'm also entirely dairy free for the same reasons. Generally I disagree with the industries and have chosen not to eat meat, dairy or any other by products.
However it's a choice I have made for myself and therefore will still allow my husband to eat whatever he wants. I even cook it for him. My son is dairy free but I let him have meat when he wants to and at other people's houses.
My DH though has decided enough is enough and said that the reason we aren't conceiving is because of my diet and the reason our son is so needy is because of HIS diet. I know this is a load of rubbish but don't know how to convince him that I can keep going and that I'm doing it because it's what I believe in not because I want to be difficult.
A thing to consider though is that my mum has been in his ear about how I used to be picky with food and this is a control thing. I also used to have eating disorder issues and she said that's why I do it too. I agree that it's another form of control but my weight has remained stable and it's not restrictive.