my exh was verbally very unkind when he was thrown out by me. (He thought I would want him to stay having been unfaithful for two years!!)
Anyway, a while after him leaving, I asked him why he had been nasty to me, he replied to hurt you !!. This is how some men think they can get to you. As for your family, they sound delightful.(NOT)
Moving forward, your self esteem sounds at rock bottom. You need to start loving yourself. Do things for you, I know money is tight for many, if you can afford to, buy yourself a small bunch of flowers, a chocolate treat then move onto bigger things. Get your hair re-jigged as a model at the local hairdressers, paint your nails.
As time goes on you will gain more confidence, this is something men like in a woman. Make tomorrow the day you start making yourself a priority, onwards and upwards.
You feel your partner was right saying this. He was not right, you are bringing up three children and running a home, being a mum is not easy at any stage of childhood.