wediblino, just wanted to sympathise. I can because my dd was like this. She sleeps great at 3.8 now. I am going through exactly the same with ds 8.5 month. His sleep went from bad to atrocious in the last 2 weeks.
Some babies are just like that with their sleep. I know, I have 2 in a row . They are not good or bad, they are what they are. These are their needs. Unless someone has had one like that, it is difficult to understand the feeling about being held to ransom that you describe.
I cope by co-sleeping. dd and ds were bf-ed for a long time. Both are velcro demanding babies. Forget lying in cots or buggies - they did not DO any of that. It is a temperament thing. Like they have an on switch, that won't switch off without your help.
Other posters are right about making sure he gets his sleep. If he needs to sleep on you, then so be it. If you feel you are going insane, wait a few days and see how you feel. The little b_ggers usually improve for a few nights just to dissolve your resolve.
Try cc if you wish. But at this age, I would not persist beyond a few days without improvement. Your ds2 is still very little. I weaned dd off the breast at 17 months and she finally slept through - but that was my breaking point. I am glad I got to that age. Dd could understand more. I felt I could impose limits. She was crying out of tantrum of not getting her way, rather than just fear.
Sorry if I sound a bit abrupt. I am hurrying to get ds to bed! I hope it improves for you soon. This is a very short time in your life. I promise you, it will end at some point. hth