I have 4.
The only time it's a pain is on holiday, even a "modest" holiday ends up mega pricey because of the need for so many flights, rooms etc. Our eldest 3 are ND (autistic, ADHD, autistic in that order) and oldest 2 cannot share a room despite being same sex.
I am extremely glad I didn't stop at 2. My eldest 2 would never have spoken to each other in adulthood. They are complete opposites who completely don't understand and rather dislike each other. Number 3 is the glue of the family; he gets along with 1 and 2. Number 4 is NT and the organiser, competent and confident.
Financially it's fine. We have savings, can afford uni for those going. We have never really thought that skiing holidays and top of the range cars, or designer clothes type of stuff is important. We are probably a bit scruffy. I drive a big Skoda. We mostly holiday in the UK. No one has private violin lessons - though if they had really wanted to, they could have. I don't feel over stretched. I don't think the kids feel under loved or under supported. They certainly are immensely adored.
House deposit wise - they each get 10k on their 21st birthday to use for their future as a bit of a bump start. Any of them can come home to live whenever they want. We will help with things like driving lessons. No one has bought a house yet but I guess we might help out a bit, but I kind of think that starting modest is ok. Luckily we don't live in London and if you want to buy, and can only afford a one bed flat in a less wonderful area, that's where you start. No one needs to start in a 3 bed semi with integrated appliances.