How dare he call you "silly"!
It is quite normal I think to feel under pressure whilst on ML and BF, the house will be less tidy than usual, you will be more tired and more likely to burst into tears, and arguments over "trivial" issues become more common.
However, a "normal", loving partner would be more concerned about trying to ease your load, sharing the burden, telling you he really doesn't care if the house is messy, offer to take the dcs out for a few hours to give you a break, a few hours sleep, time to read a book, have a manicure, anything to help you feel better.
You have the right to be upset, and the right to cry whenever and wherever you want without being called "silly".
You are effectively a single parent and would probably feel a whole lot more relaxed on your own.
I'm not suggesting you leave for good at this point, but I do think you need to believe that to be an option, and for him to know that that is a very attractive option for you. You mention your self esteem being low. By you believing you could actually live without this man your whole outlook will change and you may well find the dinamic in the relationship changes as a result.
Have a lovely break away from him!