It's hard, taking responsibility for yourself. I was 43 when I realised I had a whole human being to take care of... make sure they were warm, dry, fed, clean, clean clothes, happy face, tidy hair, sweet friends etc... I had to take care of me. I remember the weight of it hitting me, and what a daunting task, after stumbling through all of those years waiting for someone to pick me up when I fell, which I did, many times, in crap jobs, crap relationships, crap housing... When I realised it was all down to me, I had a big panic, but then I felt better... bolder... equal to everyone else. And life's taken a different track now.
Honestly, you could spend the rest of your days wondering why those people aren't good to you, but the fact is, they're just not, and it's nothing to do with you. They're just a bit shit, and that's up to them.
Some people are lucky and have lots of people automatically there to help them, and some people don't. Some people have it better than you, but some people have it worse.
But, I'll say it once more: you are in charge. If you want this to be a fun life, for you and your little one, create that. If you want to spend your time saying 'It's not fair!' then do that. But don't blame anybody else. It's up to you. Accept the challenge.