You clearly are the parent in the relationship and as you are not married, splitting up because having children with a child is a really stupid thing to do, is the obvious thing to do.
Dragging it out is silly.
Forget the vitriol on here.
Find yourself a good therapist to support you through this.
He is living off you and letting you carry him in every way.
The dogs are yours and you clearly take full responsibility for them, just as you would children.
He would be a useless dad, just like he is a useless partner.
You do not owe him your future so do not delay.
The house needs selling if you aren't going to buy him out.
He's the sort that you might say "nice enough guy, but absolutely not parent material, he's like having a child around, rather than a partner, nice enough fellow. It just ran its course, out of road!".
The worst thing you could do is stay out of fear, you are only wasting your time.
Do not concern yourself with other peoples opinions.
They don't have to live with him.
Don't be surprised if he is appalled at the thought of you no longer footing the bill for his lifestyle.
Don't be talked around.
You will bitterly regret it if you do.