Yes, things change but it's only for a period of time that you are really restricted. Dh and I lived and travelled in lots of less developed countries before we had dc. No we haven't been back since, but I think that's more to due just with lifestyle change than having kids (older, more risk averse, maybe related to being parents, but who knows). But ours are primary school age now and we are starting to be able to see a time in a couple years when we can do that sort of travel again as a family.
As for other things, yes, you are more restricted, but you also have new worlds that are opened up to you. I've met different people as a parent. I've learned new things about myself. I'm certainly much more resourceful and resilient than I used to be. Yes, I do miss the life I used to have in away. The travel, the nice restaurants, the excitement, but I wonder if that's more a figment of it being more of a 'luxury' now. I know when we decided to have children, I was a bit bored with all of that. Like after awhile, it does get a bit same-same, if there is nothing deeper there. We have close family who are childfree by choice and yes, they travel and go hobbies and have lie ins, but I wouldn't trade places with them. A lie in stops being special when you can do it every weekend. After awhile, it's just 'not getting out of bed' and missing out on life. Same with a nice weekend away, it's lovely to do now because it's special, but at a point, it stopped being so lovely because it's just staying in a nice place and eating nice food - which I can still do as a parent, but I'm not missing out on all the other really meaningful stuff I have in my life.
That doesn't mean it's not challenging to have children, but I think your worries are pretty normal.