Sorry to hear that you're not getting the support you should from those around you, that can be really tough.
When I had DS 1 & 2 my Mum was around (we lived as an extended family) and was a huge support and influence, she was my best friend.
Fast foward a few years and sadly my Mum had died, I had no contact with any of my family anymore, we had to sell the family home and move and I was pregnant with DS3.
I wanted my baby very much but to say I felt daunted was an understatement, I had never had to manage with just my DH and I and no other support and whilst I wanted us to cope I wasn't sure how it would all work out.
18 months on DS3 is 13 months old, I'm pregnant with DC4 and can honestly say we've never looked back. We're a tight unit and manage really well, the kids are happy and don't go without and everyday I'm proud that our success (for want of a better word) is down to no one but us.
Sorry for the long post, what I'm trying to say is that even if you can't see how it can ever work, it can, and that in a years time you might be looking back and wondering why you ever doubted yourself.
We also had loads of negative comments when telling people about DS3, just ignore them.
Hope this helps.