I had older parents growing up, to be honest I did find it made me stick out a bit but they were quite old in their outlook - I think my dad would have been middle aged in his 20s! Despite that I have gone on to be an older mum myself - I will be nearly 41 when DS2 is born early next year. DH is 50 :)
I am fit and in far better shape than I was in my late 20s so rushing round after DS isn't a problem. None of my older-mum friends have commented that they feel more tired than when they were younger either. DH has other older children and has said that he thinks in some ways it is easier when you are an older dad, workwise - new fathers in their early 30s will often be middle managers so getting enormous pressure from above and below in addition to the pressures (lovely though it is) of new fatherhood. Of course he is working harder than ever now to provide for his growing brood but he doesn't, for example, have to work away any more.
I have had two miscarriages, at least one due to a chromosomal problem caused by maternal age. With this pregnancy I have had a feeling that we have somehow pushed our luck - but that is because of our own history.
I do think that you need to go into it with your eyes wide open though (sounds like you are), because of the increased risks due to maternal / paternal age, and their implications eg I knew about the increased miscarriage risk when we started trying for our first but hadn't really thought about the practical side of that - how much time we lost when I had a missed mc, then left it a few months til I felt ready to try again. Then I had another missed mc between DS1 and this pregnancy. I had "assumed" I would have had my family complete before I was 40 by popping out a couple of babies in quick succession in my late 30s (I know, I know...)
On the other hand my best friend at school had a baby at 16. Her DD thought her mum and I were hideously uncool and annoying when we were both about 25 :)