Well, you know what Xenia would do, faced with a family rebellion - she would simply say "why are you not criticising dh for going out to work and 'abandoning' his child?".
For me personally, I have to work f-t because my dh is chronically ill, so I have to earn the cashola. But even without that constraint, I would want to work p-t because I genuinely enjoy having a life outside being a SAHM. In no way do I expect everyone else to want the same choice, male or female. Dh much prefers to be at home with ds than go out to work. So that's fine (or at least would be, if he were well enough to look after him f-t, as was our original intention).
I have a friend at work in a similar position to you. All of the women in her family have been SAHMs since their kids were born, and she can't really contemplate a different option, so is holding off on having kids as a result. (She's only 24 so this isn't a major deal).
You absolutely need to follow your own needs on this. We all have pre-conceived notions about what parenthood will be like, and what kind of parents we want to be. Inevitably we are wrong. The best thing about your situation is that if you find you don't prefer combining work outside the home with motherhood, you can always change your mind.
You are not a bad mother. You're just looking for the right combination for you and dd.