Yes, I left my DD age 2.5 with my exbf.
Initially, I took her with me and rented a room elsewhere. We shared her back and forth for a few weeks. Then he came to me one evening and begged, saying that looking after her was all he wanted to do with his life, so I agreed to sign over the child benefit to him, so he could claim benefits as a single parent. I then became liable to pay him maintenance.
There was an important factor in this decision: when his parents divorced when he was about 9, his father disappeared from his life and he didn't see him for over a decade. I knew if I took DD, her father would copy his father and disappear from her life. Then she would grow up not knowing her father.
Whereas if I was the non-resident parent, I would always be in her life with regular contact and nights spent at mine.
Also, in those few weeks when I had her alone initially, I realised I was not cut out to be a full time single parent.
This is probably very different to your situation OP.
Some questions:
Does your DH actually want to be the main resident parent? Will he give up work, or will he be responsible for organising childcare around his work?
Do you want to be the main parent? Why can't you take the children with you temporarily while you sort out housing?
Will he expect you to have them 50/50, but he keeps the current house?
Unless you actually want him to have the children, don't leave them with him.