His mum is not an 'extension of him.' My ex tried this. File for maintenance. Don't bother listening to his 'so there, shut up' arguments.
I understand wanting your child's father in your and your daughter's life. I've been there. You take all the abuse and keep engaging because you keep hoping for a better outcome and it's less lonely than being on your own. I put myself through so much unnecessary shit before putting my foot down - like you, answering phone calls and allowing him to scream abuse at me, spending holidays together where he treated me like dirt, allowing him all the perks while he abused me, contributed nothing, and made me question myself and feel unreasonable for asking for the bare minimum.
But ultimately no an abusive parent is not better than an absent one - that's nonsense and an excuse. Your child will be hurt by him and his effect on you. Your self esteem is being damaged - your question on here was 'is this neglect?' suggesting that you're taking all his abusive shit onboard as legitimate and questioning yourself and taking on all the anxiety he's willing to inflict on you.
Your daughter will grow up seeing you as a hurt and damaged person constantly seeking her dad's validation - do you want her to learn that that's her role in life, her role as a woman and in relationships? Or conversely do you want him to turn her against you and both of them treat you with zero respect?
It took me a good bit of time and therapy but I managed to separate with DCs' dad and maintain proper boundaries, trust myself, and immediately shut down any disrespect. It hurts because you have to let go of the fantasy and accept that they are not the person or father you wish they were - which is especially hard when your hormones are everywhere and you're feeling so vulnerable.
But you're stronger than you know. Look for support outside of him - baby groups, family, try the Peanut app to meet other mums. You can do it for you and your daughter. Good luck <3