So sorry, he sounds really awful and I can't imagine how terrible that is to live with. It does sound like he doesn't care about you or the children.
You threw a baby hat at him... not a knife, while you were dealing with all the mental load and doing all the physical labour. Please forgive yourself, this was not an awful thing to do, but I expect it was a handy catalyst for him to disappear to the bar and have a day off. The narcissistic arsehole.
If he doesn't come back to sleep, then I would be tempted to phone the front desk and let them know what has happened. You will probably cry while telling them, and that's OK, you have had an awful time and it's expected. But do ask them if you can have some help at breakfast, lunch and dinner. I know that the staff are busy but I should imagine a staff member would feel very sorry for you and try to help.
Going home is of course an option and that would give the narcissistic arsehole a nice holiday on his own.
If I had heard someone who had been through that while I was on holiday I would gladly hold the baby throughout breakfast, lunch and dinner for you at least. Tell as many people as possible, cry and let that emotion out. This is a terrible thing to happen, it's not normal and good people will help if you ask.