@justalittlemilk
I hope to help here and provide genuine advice. I believe whether your DH means to or not, he is the one being unreasonable in this situation.
On one hand, he has to spend time with his other children but he should integrate his DS and DD into your family so you can get to know each other better. I was very much on the fence about this situation until I saw that you said your DH stays in another house with his DS when he has him. That set-up is just weird and how long does he plan for that to continue?
I think you are best to sit your DH down for a chat at the beginning of the week when your husband is home after work. In my experience, it is not about what you are dealing with that's the problem or what you say but it is how you say it that will determine the outcome.
You need to to be like this First of all, I want to tell you that I love you and our family and I am so very happy that we are together. You are a great partner and father and I could not have asked for a better husband (complimenting him in this way softens things, it is very diplomatic and helps him feel comfortable).
Having said that, I am sorry but I feel I have to say something. As much as I would never want to get in the way of your relationship with your other DS and DD and I appreciate how challenging this must be on you, I do feel uncomfortable with the current set-up of how you spend time with your DS at the weekend. I would never want to replace your DS mum as he has a mum and I respect that but I am his step-mum whether you like it or not and our son is his brother. I think I can bring something positive into his life and for our family to connect in a positive way to the best way we can
I want to love your DS as my own and get to know him as he is an incident child and he is part of you, the man I love. However, I feel I cannot achieve this as you stay in a different house when you have him and you do not bring him home to meet me or his brother. As much as I am not expecting this routine to stop straight away, I would like us to work towards creating a new dynamic that suits everyone. Perhaps he can stay here with us and you can take him out for some father and son time on the Saturday during the day and then movies at home for all of us at night and other family activities on a Sunday
I do not want to cause any problems but no matter what way you look at it, I am not happy with this situation and I would like you to go away and think about what I have said. Nothing has to be decided right now. I say these things because it is lonely for me also to spend weekends by myself when you already work such longs hours during the week. I miss my husband and our son misses his dad. I hope you realise that no matter what I come from a place of love and I do not want to cause arguments or upset for anyone. I want you to have a very good relationship with your other children, I just want myself and our son to be apart of that to some extent
I would perhaps write all of this in a letter that he can read, you can get your point across and as you are being nice about it, hopefully he will understand what you have to say. If this does not work then he is only thinking about himself.