I was the other way round from you. I desperately wanted a second one, for my son's benefit, not mine. I loathed being pregnant, hated the first few months of motherhood, didn't really feel I loved my son until he was about 18 months old. Probably mostly for the same reasons Frankiecat gives, awful birth, PND, and a less than understanding husband etc.
But, I really didn't want my son to be an only child, particularly with being a forces family, and having to move him away his friends every couple of years, so I knew it had to be got through. My husband however, really doesn't get on with his brother even now as adults (understandably so, the guy's a twat), so could see no benefit to siblings, and didn't want me to go through PND again (he's learning understanding slowly).
Cue several years of argument, me giving up the argument as too near to 40 now, when I accidentally fell pregnant. And now at 7.5 months pregnant, and too late to do anything different, I'm terrified that this is a major mistake. But, I know that this is just me being scared about the next few months, tired, hormonal etc. My son at 4.5 is VERY excited, and looking forward to helping. We'll see how long that lasts! He had been asking for months when his sister was coming, because all his friends had them.
In summary, I think we all have inner conflicts about whether our decisions are the right ones, and those conflicts can change from day to day (in my case, at least)