Biggest problem is that one of you will probably love it more than the other, so one will always be nagging to get down whilst other can't face the journey. That will be frustrating for both of you.
If you need a plane to get there then you need to factor in the cost of getting there - flights are hellishly expensive on a Friday night (ditto Sunday night return), wherever you go. Regardless of cost, airports at these times with kids are horrible places. Fairly likely that you will be hauling the kids down there by yourself, and DH will swan down by himself on late flight (and you will have to collect him at other end). If total journey takes more than 5 hours then I would be surprised if you end up there more than once a month, max. Which means that the first half day will be spent restocking fridge and getting logs etc; last half day is washing and drying sheets and trying to clean everything up. Bear in mind that in the UK half the time it will be cold/raining, and not all of you will want to put on boots and brave the weather. If there is a nice village pub, one of you will slope down there, leaving the other to put the kids to bed.
It's not too expensive to maintain in terms of utility bills, etc, but you need to factor in insurance (forget contents, as the cost of cover won't be worth it because it will be classified as unoccupied) and household bills. You will also need to find a gardener to keep it going (expect to pay around £50-£70 a week between April-Oct for an average garden). You will also need to find a keyholder locally you can trust who can clear away post and go in if there is any type of emergency.
People locally will know when it is unoccupied, so burglary is always a possibility. Things like 'bin' days can cause problems too - your keyholder will need to put your wheelie bin out and back in after you leave or else everyone will see it left out...
Once there, how will you travel? Will you need to leave a car there? That will be expensive - especially when you arrive at 11pm to find the car won't start.
I have had a second home in the UK which was a dream because I could get there from Central London in 2 hours, so it was never a chore - I could always nip down every week so it really did feel like a home and I made good friends in the village. I now have one in France which is around 6 hours door to door and although I love it, the cost of flights and hire car make each trip comparable to taking a 'proper holiday' cost-wise, and it really is a slog cleaning it all and changing sheets at the end of each trip - especially if you are only there for four days - the clean-up takes just as long as if you had been there 4 weeks. I used to have a cleaner do it for me but was effectively paying around £150 per visit, once the sheet-cleaning service was added on.