Do you leave him with something to keep him busy? If not, try something like a stuffed kong or a treat dispensing ball.
The more you do it and he gets used to occupying himself while you are out of the room the more he'll become relaxed. Ideally, pick a treat toy that he only gets when you leave the room and remove it as soon as you return. That way he'll eventually start to look forward to you leaving so he can get his paws on his favourite treat.
My two have a treat dispensing ball, kong wobbler and kongs stuffed with their usual kibble, mixed with tinned pilchards/sardines in tomato. When you eventually build up to leaving him for longer periods you can freeze the kongs in advance to make them last longer.
If going out of sight is too much for him, try just sitting quietly, slightly in view, on the other side of the baby gate and work up from there. You should start by only leaving him for literally a few seconds and gradually build it up in say, 5-10 second increments. Once you get to a few minutes you can increase the size of the increments and the longer he can go the more you can increase the increments. If he starts whining again at any point, go back a couple of steps and start again from there so that he's always kept under threshold. It's a pain, but most pups do get past this stage fairly quickly if it's handled right.
I do understand how frustrating it is by the way. I have a 16 month old rescue that has major abandonment issues due to his very early life and it's taken a long time to get him used to the fact that the world doesn't end when I go upstairs - I still can't leave the house for longer than an hour and even then only if I've left enough kongs and treat toys to distract him for the entire time. Your boy will pick it up much quicker, as he won't have the deep seated psychological issues mine has.