It depends on you and your children. And if you work or not, how much free time you have, how much money you have. If you have a lot of time and money then surely having 20 kids would be a doddle. If you are skint and work 12 hour days then having one would be hard!!!
I found 1 to 2 a complete doddle. Really lovely and easy. 2 to 3 hard.
Its the practicalities. Getting 3 in a pushchair for a decent fast walk. Getting three car seats across the back of the car (had to change car as our very large estate couldn't take a booster and two seats!!!). One child is always saying they are left out if we are messing around or playing. Holidays cater for two children, two adults often and it is expensive to allow for another. Staying in a huge number of hotels, B&Bs is a nightmare as you are only allowed 4 to a room so have to book 2 rooms if going to a wedding etc - travel-lodge and premier inn don't allow three children to a room and so if you travel up and down the country visiting relatives it becomes more difficult.
Now they are older getting reading done with them is a battle time wise after work, along with homework, etc. I spend my life on the road picking up from play-dates and clubs and this increases with the more children you have. They are squashed in the back of the car and moan and elbow each other!!!! And, importantly, in the case of childcare - I have found people reluctant to take three (it just seems it is one too many for most people to cope including grandparents).
I do envy other people with two children if I am perfectly honest, although I love mine dearly and certainly wouldn't change it now. I also worry about how to help them out financially (three harder than two) and that (if) they inherit less each when we die (if it hasn't all gone on care home fees).
It has been a massive strain on my dh who finds it very very hard. I do feel that we are giving the children a worse childhood than they would have had if they were either a single child or part of a two.
Obviously all the above is just a personal experience.