Sorry OP. I've been where you are - not so much in that I wanted to start a family (although i was in my thirties and childless) but in dating a man who I really fell for but he seemed very luke warm.
I felt in a state of permanent confusion. It felt like he was doing the basics - showing up for dates, having sex, meals etc. He had been very enthusiastic at the start and then when I got keener (and to my shame a bit needy) he withdrew.
We then ended up with this very unsatisfactory 'relationship' where I had strong feelings for him and he kept showing up for dates etc but didn't give me any vibes of being enthusiastic about it. It was utterly heartbreaking. Like you I tried to find out what the heck was going on and just got answers like 'I wouldn't be here if I didn't want to be'. He was having a hard time with work too and so it may be that I didn't wait long enough. Like you I sort of felt forced into ending it as he was bringing some of his problems into the relationship and making my life more difficult. That on top of the fact I felt he was luke warm forced me to end it but like you I really, really didn't want to. I wanted him to love me the way I loved him and it was utterly heartbreaking.
I then went out with a man who I kept my cool with and of course he fell head over heels for me. The difference in the behaviour of the two men was like night and day. Man 1 turned up, said the right things but left me feeling very vunerable, needy and unloved. Man 2 was obvious with his enthusiasm, love, feelings, putting effort in, organising things he knew I would love etc.
(I didn't fall for man 2 and we did break up after 2 years but that's another story)
Anyway the point of my post is I get it and you can be pretty attached to someone after 4 months. Breaking up with someone you don't want to is very, very hard.
Me and man 1 actually tried again several years later and on our 2nd meet up I felt so bad inside I actually wanted to burst out crying. So I told him we weren't compatible and never saw him again. He reached out to me a couple of times but I didn't respond.
My advice - if you think you made a mistake (and think long and hard about it) then ask him for another chance. If he says no, well you have to move on. If he says yes, give it a bit longer and then say by xmas evaluate it again.
I've realised that sucessful relationships need two people who are attracted to each other and have the same values etc but the timing also needs to be right. If one is very much in a relationship place and the other is distracted with work, illness or whatever and not really focused on finding a relationship it doesn't really work.
To this day 11 years later I still think about man 1. I was very attracted to him physically and mentally which is rare and for a long time I felt we were meant to 'be together'
I never went back though and I still feel sad about it.
I wish you luck whatever you decide. Just wanted to let you know you are not alone. I've been there and I've also been on the other side (where the man's heart was broken). Hugs