Bingblang so can living together 24/7, just pop over to the Relationships page!
Op, we live exactly like this, my husband works away, not always in a stable pattern. For those saying it is not romantic to live away from your partner, this hasn't been our experience, we love our weekends together, it's far more romantic to have a night together on Fri and Sat not having seen each other all week, there's lots to chat about and we both make the effort to spend time together both as a couple and as a family. It works well for us, anyway.
As for being unfaithful, lol, again see the Relationships board if you think someone living with you stops them being unfaithful. We both work, we both have ample opportunities should be be of that persuasion. We are not.
The downsides are that you can sometimes feel lonely in the week, all the childcare responsibility is onto you, I use the time to work hard at my career but I can imagine it could be quite dull waiting around til he gets back.
Although, given your husband gets in at 9 at night anyway, I'm not really sure it would look very different than if he lived at home and worked long hours. And 3/4 days at home a week is more than ideal, you are very lucky to have this set-up (so 7 days together, 7 apart).
As for the children, this is the main reason I won't move for his work right now- it is untrue to say children are just adaptable, they might be once, but you can't keep shifting them around without taking into account the toll in terms of friendships and general unsettledness, I have found this the worst aspect of moving around in the past which is one reason we are not moving now.
Good luck with your decision.