We both work full time and it is difficult to get the time off you need. It is easier when children are older as we did not have to cover school holidays as they were left alone during some of the holidays, went to clubs, shared childcare with friends.
You could get things relatively cheaper if you book well in advance, not go for AI etc.,
It also depends on the place you are travelling to. Some places are expensive to get to, but cheaper to eat out etc., when you get there. I completely understand where you are coming from, I love travelling too. In fact, I think it is in my genes - my grandmother had traveled all over my birth country , my mum is in her 70s has travelled all over the world. I am on holiday now, but already thinking of the next one !
We travelled to Atlas, anti Atlas , walked in one of the valleys ( beautiful, lots of fruit trees, streams ) , visited a Bedouin place, women charity which runs shops to sell Árgan oil, camped in Sahara after a long walk on Camel. My 40th birthday I woke up seeing sunrise in Sahara sitting on a sand dune- blissful !