Long and rambling response - but something in there may be of help I hope!
My DD (2.5) has been like this lately too. She has always been dreadful at going to sleep but used to sleep through. Then in the last two months started waking every night and screaming and crying until we took her in with us. I was happy enough with this solution (quite liked cuddling her in fact) when she was happy to actually sleep in our bed. But then she started wanting to play and talk and wriggle all night long and no-one was getting any sleep so it had to stop.
Two days ago we put a plan into action which so far (fingers crossed, touch wood and anything else you can think of) seems to be working. Last night she went to bed around 8 and slept through in her own bed til 8 this morning.
What we did was to talk to her A LOT during the day we had decided to start trying to get her to sleep through about how we had a plan to help her to sleep through the night in her own bed. There was a lot of "isn't that a good idea?"and "won't that be great?" and "you really are a big girl now"- we also talked to her alot about how things would work when she woke up in the night - that we'd come and make sure she was ok, and tuck her in but that she would have to stay in her bed and sleep there. She was a bit taken aback but agreed. Then when she did wake up at 1oclock I went in and did what I'd said - of course she hollered and shouted and wanted to get out of bed but I was really firm. I stayed with her for a few moments and then left the room, having gone through the checklist of things she likes - nightlight on, door open. The big change we have made is to leave her nightlight on all night - does your DS have one - if not might be worth getting one. Or if he does have one but it's not working, maybe that'swhat is keeping him awake and you should think about darkening the room.
Is he warm enough? My DD kicks all the blankets off so I now put her to bed in layers and layers of clothes, culminating in a fleecy sleepsuit with feet in it. Is he too warm?
I'm sure you've thought of all these things.My heart goes out to you - it is really rotten.
In terms of going to bed at night, DD used to scream and shout and then hop out of bed and come downstairs until I very calmly told her that if she did that again I would have to come up and turn off her night light and close the door. She now stays in the bed at least (and screams and shouts!).
What time do you start the "bedtime routine" ? In the last two nights, when DD has been much more settled, I have actually started much earlier than normal, going upstairs between 6.30-6.45, it used to be 7.30ish. I think this may have helped too - getting her in before she is overtired.
What seems to work for us (famous last words) is to decide a plan of action and tell her what it is and then stick to it,really calmly.
Best of luck - I think it is all just trial and error but conistency is key (extremely hard when you are exhausted yourself I know)