This sounds exactly like what my dad would have done to 'get at' my mum in an argument.
It sounds like your partner, like my dad, is a person who puts the importance of his kids feelings below the importance of punishing you in an argument. I know you said they're too young to realise the card isn't there, but are they old enough to have been upset if they saw it in the bin? My 2.5 year old is only just old enough to do some scribbles in the card and he'd be devastated if he found it ripped up.
Assuming even that they're literally babies and wouldn't have a clue - are you to believe that he will stop doing things like this once they are old enough to know? Because that happens sooner than you'd think. Are we saying that once you have a 2 year old who can scribble a black line in the card, he'd never dream of ripping the card up?
I highly doubt it.
What kind of person can rip up a picture of their kids smiling faces? Could you do that, honestly? Go and look at a picture of your kids. Could you feel comfortable just ripping it up right now? And then putting the pieces on top of the bin?
I'd get out of the relationship. There is no amount of compensating you can do as the sane parent to make this kind of thing ok and have no impact on them.
I put people like him, like my dad, in the same category as the kind of men who abandon their families and are only interested in a relationship with their kids if they can 'keep' (or win back) the wife too. My own mum's dad was just like this. He raised her for 7 years then abandoned her, never to be seen again. He had no interest in seeing his kids unless my grandmother would take him back.
I know that might seem like an overgeneralisation, but this kind of coldness and nastiness sends my spider senses tingling. I don't trust a person who sees their kids as an extension of their wife to the point they do horrible things that would upset the kids with the purpose of hurting the wife. Something's broken inside these people. He can fix it himself in his own therapy if that's what he wants, but he can do it for himself.