I'm actually really gobsmacked by your dp's reaction. That's just not being a parent, I'm sorry. Fine to FEEL embarrassed - not fine to just walk off.
Do you both have an agreed strategy for dealing with toddler tantrums, btw? Personally I am not a fan of extended 'negotiations' and explanations with toddlers when they are in a state, I just don't see it working, they don't have adult reasoning skills, and parents get just get even more frustrated and upset. In any case, shopping trips or public places make everything worse, and if you don't have an agreed, shared approach, it's going to be very hard.
If you are both on a shopping trip with the kids (would avoid that, myself, but whatever suits!) - then the technique of 'remove upset toddler from shop physically' works fine. The parent left behind can carry on the shopping bit while the other parent takes charge of a wailing toddler outside.
Otherwise, something like a 'one reasonable warning/request' followed by 'direct action' works (child is asked to put something down and come - they refuse - go and take object off child and pick them up' for example). That takes the 'oh, god, what do I do with them' element out of it all, because you have a strategy - and if both parents stick to it, then it avoids that difficult moment when one parent takes over from another, as if they can 'do it better'. Which often causa another argument....
All that said, I do dislike, intensely, the too cool for school attitude of a dad who won't go to a dad and child group, or who gets embarrassed when his friend sees him with his own child misbehaving. Kids will be kids, and they need a dad. He needs to step up to the mark. And if he won't pay any attention to childcare books or advice, then he's just being very selfish and letting his family down. Including you.