I have a friend who left her husband under similar circumstances and went to live with her parents. Her husband filed for an order with the courts the next day to try and have the children returned to the family home, as my friend had taken them about a 5 hours drive away from where they had been living as a family to go to her parents. She went with the intention of relocating there permanently, which she has managed to do (her kids were pre-school age at the time). My friend then proceeded to spend over £100k in legal fees fighting his attempt to stop her relocating, agreeing how they would share care and then agreeing the financial settlement she would receive from the divorce.
Despite the distance and his work commitments the kids are with him every other week for 5 days (so 9 days with her, 5 days with him on a fortnightly basis) with school holidays split evenly.
I say this as if you think your split might mean you relocating to be nearer family or other support, brace yourself for the court action he might take and might action very quickly. And maybe think about taking the action first, but if you have no access to money whatsoever I'm not sure this is a goer as I'm sure it costs something to put the court order in.
Also if you do ever get back into the house, (which might not happen unless he's more reasonable when sober and lets you in, as you say you didn't take your keys) and you have any pets, take them with you if you want them and they are yours rather than his or you'll likely never see them again.
As others say, you won't lose your children entirely, but a shared care arrangement with your ex will most likely happen. I guess that could feeling like losing them in a way? My friend had years of evidence of coercive control, verbal domestic abuse and even martial rape and the courts didn't seem to give a crap about it and still allowed her ex 5 days a fortnight with the kids.
Don't mean to scare you, just prepare you!