He sounds a bit like my h. I too am not happy with my weight and he knows it. I think he is dissapointed because he knows i am happier when i am slimmer.
I wonder if just wants you to be happy and he can see that you focus on your weight and is just not very good at expressing it.
He has wanted to be with you whatever your size - even though i know you say he has pushed you away over your weight.
Maybe you are just picking up on that as it is what you really feel about yourself.
I can really relate to this and it may be worth discussing it at your counselling.