Does he really make the whole day miserable? Or is it more, that there are a few good bits and some really terrible bad bits? I had to learn to tell myself that even a couple of almighty tantrums and a lot of whinging did not spoil the whole day, there was also the fun bit where we watched the show or jumped on the trampoline or fed the ducks no matter what horrors happened before or after.
Also I found my DS was not good at whole days out or big exciting trips. He was much better with a fixed routine that included some shorter outings - half days at most - or simple things like going for a swim or a play in the park or a trip to the library or a snack in cafe. Just one thing at a time. Trying to do big day full of exciting things tended to cause trouble.
The memorable occasion when DH said to me "you're not supposed to enjoy family outings you're just supposed to have them". That made me laugh and somehow the rest of the day went a bit better, I have a happy memory of DS and DH soaking wet going down the helter-skelter in the rain.
And every single holiday we have ever been on, the first day I think "why did we bother, he is miserable and foul tempered and he hates it and we are all having a horrible time". But after a day he kind of settles and we do all start to enjoy it.
Sorry you're having such a hard time. One of things I liked abut the parenting group was meeting parents who were going through the same things, or something different but just as bad, or worse. And you're all figuring out ways to cope together. You'll get some ideas, some will help, some wont, the ideas that do help will be like gold.
I agree that it isn't helpful to tell the OP to see the GP
My GP pointed me at the parenting group (and other useful things). Helpful much! And if the OP's DS is always miserable then maybe he needs some extra help too, the GP can help with that too.