4 DC aged 6 and under must be shattering! I know how I felt when my two were that age and I was a SAHM. You will, no doubt get a lot of posts telling you how nasty/twattish (and worse) your DH is. I suspect though, that he is seeing your time and energy, going to the children ( as it should), and is wondering where he fits in. Some people, men and women, see sex as proof of love. Maybe he is one of theses. I would say that actually, 1 or 2 times a week, with all that you have going on, is pretty good! Do you even want to have more sex? If so, then he needs to stop whining take a pro-active approach himself.
Does he do enough around the house and with the children? Someone on here suggests that work, chores, childcare, should be balanced between you so that you each have the same amount of 'leisure' time. Does that happen or does he get home from work and think he is finished for the day whereas you still have loads of stuff to do?
If he wants to leap into bed with you in the morning, is there any chance that he can arrange a weekend away for the two of you? Of course it would also be down to him to arrange suitable childcare as well.
Ha she ever looked after the children for the whole day? A weekend? maybe he should. Could you have a weekend with your family/friends and leave him to it? (Really, make sure he has to do what you would normally do, washing, shopping etc, won't work if you leave him with a house/freezer full of food and a pile of neatly washed and ironed clothes).
If this really is the only issue between you, it may be simply that you have to 'educate' him so that he understands what your life is like and how you feel over this.
Of course, it may be that he is doing all this and that you just don't feel you want more sex. In that case, you need to tell him that. You need to explain that fore the moment, your time and energies have to go to the children, but they are only young for a comparitively short time and as they get older, it gets easier to give time to each other again. Honestly. When mine were little my life just seemed like a blur of nappies/prams/playgroup/washing/ etc etc. As the DCs grew older and slightly more independant, things changed and gradually I realised they didn't need me in the same ways as when very young. They never stop needing you, but not always in such a 'hands - on way. Now mine are late teens/early twenties, though they are still here ( well, one at uni but home for holidays etc), it is more like having housemates and much easier all round.
Good luck. 