I've let him physical abuse me..cheat..lie..run between me and another girl 4 times..disrespected..no financial or emotional support and now this.
Well this puts in context. I don't want to be harsh but really why do you say "and now this" as if it's a surprise?
Someone who has been treating you so horrifically, isn't really going to break up with you and say "darling it's me not you, you are perfect" are they? He has been abusing you in all senses. That sort of nasty commenting is exactly what you should expect from a shit like that.
Just walk away and be glad he's out of your life.
As for the "not phyically attracted" thing, it sounds like this is a trigger point for you more than usual because someone else said it to you before, but really you need to focus on how you feel about yourself.
Think about it - do you find all men physicallly attractive? No of course you don't. It's a personal thing. Sometimes it can be physical (big changes - weight loss or gain, hair loss in men even a woman dying her hair to a different colour ) can mean that attraction can come and go. No one in the universe is physically attractive to all members of the opposite sex. No one. For every hot Hollywood bombshell, there will be a man who can only get turned on by a curvy woman, a black woman, a white woman, a blonde, a brunette etc.
So the only thing that really matters is how you feel about yourself.