I think you’re doing the right thing op. I suspect your DS didn't realise the seriousness of what he was doing at the time. However, now it's done, I think losing the party is a suitable consequence and you're right to follow through. I really feel for you and I think your two-pronged attack of cancelling the party but stepping up the 1:1 time is the way forward.
For what it’s worth, your DS is not the first 9-year old to act up because a new baby/toddler is suddenly running the show, and he won’t be the last. Of course, that doesn't make it ok for him to be nasty. It just means these things happen to the best of us and I hope you’re not beating yourself up. It's what you plan to do next that matters.
As you say, you’re not cancelling his birthday, you’re just telling him he can’t have his friends over. This seems entirely appropriate given that he’s shown unkindness to other children recently.
I find claims that this will “scar him for life” or that he’ll “resent you forever” more than a shade melodramatic. In fact, if he’s genuinely remorseful and you ask him outright, I suspect he’d agree that not having friends over is a fair consequence. It doesn’t mean his birthday will be miserable – just not exactly what he wanted. Children who would “resent you forever” if you remove a privilege because of their poor behaviour are children who need to work on their sense of entitlement. They are entitled to have their emotional and physical needs met. A party is not an emotional or a physical need - it's something very nice that they might be allowed to have if they've earned it.
I agree that a behaviour book could be a good idea, but can I make a suggestion? It’s something that I’ve seen work before: You divide the page into two halves. One half is for basic behaviour – following instructions, abiding by the school rules. You want to see that half of the page empty except for the words “all fine” or “no problems”. If any silly or poor behaviour is mentioned here, you will be disappointed; there will be consequences. The other half is for acts of kindness. This could be helping the teacher, helping other staff, looking after little ones if they’re upset at breaktime, tidying up if he sees it needs doing, etc. This is the bit you’re interested in. You want to see lots of writing here. Maybe he can write himself the kind thing he did so long as the teacher or member of staff initials it. (Busy staff members may appreciate this!) Hopefully this way he sees it less as a log of his bad behaviour and more as a “good deed diary.” (Also, if he's been mucking about at breaktimes at all, it might not hurt for him to look for something constructive to do to keep him occupied.) Personally, I wouldn't be promising rewards for good deeds - just show a lot of interest, praise anything good, ask him to tell you all about it, and tell grandparents/friends/relatives about it loudly and proudly when you know he can hear you. Then at a later date, tell him how proud you are of all the kind things he's been doing recently. If you want to give him a reward at the same time, fair enough. But it's not a case of "one good deed gets you one Mars Bar." That's not what it's about.
Regarding 1:1 time, I know it’s really important to do fun activities together. However, the fun times aren’t easy to arrange if you've got a 2yo to think about, and they don’t always allow for proper reflective chats. Do you have a memory box from when your DS was small that he could help you to organise? Or some photos that he could help you put into an album? This would be a great way to remind him that he is special to you. You could talk about the sort of things that he got up to when he was small, and compare with the sort of stuff his brother gets up to. This might allow you to test the water a little – see how he feels about his baby brother taking up so much of everyone’s time. It might be a good time to think of some things he’s done recently that you thought were really grown up or smart and tell him how proud you are. It might be a good time to tell him that you miss spending time with him on his own, and to ask if he’d like to arrange to do something fun together.
Even just doing a few chores together (preparing food, making beds, washing dishes) can be good bonding time. It’s a good chance to let your DS feel helpful, successful and appreciated, and you can chat about whatever he wants to chat about while you’re working. I sometimes think this sort of quality time is more valuable than trips to theme parks or whatever. It’s certainly more doable on an ongoing basis.
Whatever you decide to do, fair play to you. A few posters seem to think that you can’t be both strict and loving at the same time, whereas I firmly believe the two go hand in hand. Hope it all works out!