You don't know what life's going to throw at you. When it comes to having kids, even if you have the basics in place then there's never a 100% "right" time. You'd always find a reason why you shouldn't do it at whatever age you consider it. Anything could be round the corner.
You could decide to wait a few years and then before you have the chance to even try for children, your DH might drop dead (god forbid).
You could end up trying at 35 then struggle to conceive and then the older you get from that point the more difficult IVF could be. Imaging getting to the age of 40 having waited for your friends to start having children so that you wouldn't feel the odd one out/too young, then you all start to TTC and you watch your friends get pregnant no trouble while you end up having multiple rounds of IVF and going through all the stress that brings. Maybe it would be unsuccessful.
You could have a baby at 37 then develop a terminal illness when you're 50 and leave them without a mum when they're mid teens, as opposed to mid 20s if you'd had them when you were 27.
You could end up progressing in your career over the next few years and find it harder to combine that with starting motherhood, and wished you'd had the children first and THEN been able to devote time to your job once they were becoming more independent.
You could get to 40 and feel GLAD you waited till then, only to have a baby and find that you're exhausted by the time they're 10, with the stress that menopause and having aging parents can bring. And you then hit the teenage years with all that going on. Not a good combination.
I don't think you should overthink it. Mother nature is urging you to go for it. Your DH feels the same. You have everything set up to provide a stable life for a baby. What's stopping you?
Someone has to be the first in a friendship group to have children. Why shouldn't it be you?
On the other hand, you could have a baby with extra needs next year, and feel the strain of that while your friends are off to shows and on holiday and wish that you'd had a few more years of freedom.
There are downsides and upsides to all possible age of having children. Nothing is certain. All tht matters is that the child is love and wanted by you both and you can provide for him/her.