He lives at mine and he has nowhere else to go.
That is not your problem. Honestly, it isn't. You've made it clear to him that you're willing to be a doormat and so he knows you wouldn't throw him out. Which is why he feels comfortable treating you like shit.
Being miserable in a relationship is much more lonely than being single. This isn't an exhaustive list, but love is:
feeling safe with that person
feeling happy when you're with them
respecting each other*
supporting each other*
making time for each other*
(*note, each other - not all one way)
From your OP you couldn't tick a single thing off that list. it's time to get rid of him, and where he goe is his problem to sort. It's your place so as soon as he's out change the locks and block him. You'll hurt after you've done it because you're so deperate for someone to love you. But it isn't him.
Therapy, therapy and more therapy to unpick your childhood, why it makes you so deperate, and why your self esteem is on the floor (answer to the last one - because you spent your formative years being told, in actions and/or words, that your parents didn't love you, and you internalised that and formed the belief that it was your fault for not being loveable. You were wrong but it will take you a long time to absorb it.)