My 3yo is pretty well behaved but I wouldn't let her walk around the supermarket, always put her in the trolley unless it's just a quick nip in for milk and bread. Not putting him in the trolley is just giving yourself needless hard work. If he has a tantrum when you put him in the trolley then sit him in, say a firm "no you need to sit in the trolley" and wait outside until he's calmed down. Don't engage, don't try to reason with him, just wait until its passed.
If you have said no to something then stick with it, even if you change your mind. Never giving in to tantrums and always following through on consequences for bad behaviour. Give realistic and immediate consequences, so no ice cream if you mess about in the restaurant, or we will go home if you don't listen to me. I have followed through on both of these and it was hard and I did feel bloody guilty when her friends were eating ice cream in front of her but she hasn't played up in a restaurant since.
I do think that telling him his behaviour makes you sad is pretty pointless at that age, he doesn't understand.
And don't try to reason with him, just a firm expectation of behaviour, tell him a consequence if that expectation is not met and then follow through on that consequence.
Don't be embarrassed, honestly I would feel more embarrassed for people to think I'm being a wet blanket with my DC than for them to play up but people see me discipline them.