I am not yet a mum to a child who joined the family by adoption so please ignore me if you wish to!
It may well be a competition thing because this is what he experienced in life before. Please do not say anything identifying.
So for example if DH put his leg on you or hugs you maybe DS feels 'Oh no I will be pushed out', because that is what happened before (if it did etc, or maybe it did not but he felt it did).
It may be a sign of something else so I would leave lots of chances for him to talk about stuff.
When he whispers 'I am hugging daddy.' I would say 'Good, I love to see you and Daddy hugging."
There can be daddy and ds hugs, mummy and ds hugs, mummy and daddy and ds hugs etc. And sometimes there will be mummy and daddy hugs, but because he maybe feels jealous keep it quick and low key.
If he says he wants you to move I would either ignore the words or just say, 'This is my home, I am staying', and 'I want to be with you and daddy' or something and quickly change the subject. Whatever he says e.g. but I don't want you to be here, I would just say something very light and simple like, I am here to stay and I love you both.
Try and stay calm. He may be testing the water, to see if he can make you go, because if he can that will maybe prove to himself you do not love him or that he makes people leave and it's all his fault. I have no idea, I am just guessing.
So try and shake it off, it's about him. It's not your fault and if you accidently start it by showing your hubby some love try not to worry.
Personally, I would not be overt together but occasional signs are normal and you may be able to increase when he is more secure.
If you feel strong enough I might be tempted to say when he says something mean, do you want to tell me why you say that? But be careful, what he replies may not be what he truly feels and you may divert a lot of energy answering things he does not really mean. It might be easier and safer to say, 'Is there anything else you want to tell me, you don’t need to but maybe can if you want.'
Bless you, you sound lovely.
It must be tough for you and ds to have your hubby/his daddy there some of the time and sometimes not. Is he in the army or some other service? (Don't say which, I will just say army for them all!) I wonder if other army families can offer appropriate support and suggestions of how they cope, for children who join families by adoption and for kids in general. NOT that you will copy them or it will be right to do so but it might just offer ideas of how to manage family life with a dad who is there sometimes and maybe full on some of the time and just not there other times.
I’m not saying it is not adoption related, I am just saying it might also be about his having a dad who is not around all the time and then around a lot for a period of time.