Too late now but for any future trips away, I think the best way to deal with it is before anything is arranged, sit down together, get the calendar out and agree when you will be away, and at that point agree what days he will take off, so as to give him time with the kids and you a break.
Trying to organise it now with immediate effect is asking for trouble. It should have been agreed and booked at the same time as your flights.
As for now, if he's adamant he can't take time off work then you can't force him to, but I would insist he finishes early some days and takes the kids out and at the weekend too.
Are you most bothered about him taking the kids to give you a break, or all of you spending time together? If the former, take yourself off on Saturday and Sunday and give yourself a couple of days respite.
I really think telling him calmly and firmly what you want - I need a break, so could you take them out on Saturday please - is much more effective than getting upset/angry/disappointed/martyrish (not saying this is what you're doing). I inadvertantly let my dh get away with murder for years. Didn't do him or me any good at all. I'm much better now at stating what I want and expect.
Good luck.
re the card, to be absolutely honest I'm sure my dh wouldn't have sent one, even more so if it would have arrived late, as he just wouldn't have seen the point. I would guess many people but especially men would think this, so I wouldn't be too bothered about that one.