meplusone so sorry you are going through this. You sound like you are being totally reasonable.
I hesitate as I write this because I have no easy answers.
I hope you and your sister will iron out your differences, but if they are long standing, from the past, this may be really tricky.
How old is her son?
How is he leaving messages? By text, on paper?
Why is he behaving in an unpleasant way to your daughter? Do you understand what is going on behind the scenes at all (if at all)? Is there maybe tension in her marriage of family and her son resents your dd? None of this excuses things but may explain them a little.
You said "I just feel so sorry for both my children that we have this stress and not happy wider family celebrating them, family occasions are difficult and I'm very worried about the celebration ceremony."
You may decide for the kids if they feel this stress outweighs the benefits of meeting with wider family that you want to see less of them. If the benefits of wider family outweigh the stress then can you keep the wider family contact and minimise the stress?
Could you meet in an external place, like a park or soft play, so you can leave if things do not go well and you are not on anyone's home turf?
Do they need to come to celebration ceremony?
If you just want to remain friends in the family and move on from this incident can you just put it to one side?
If you wanted to do this, what do you think would work best, meeting face to face? Can you go out to lunch without kids and have a chat? Just listen to her concerns, get her to say how she feels in her own words, without interrupting etc. Then explain in the most simple terms your situation. Then try and agree to put the past behind you?
Or send her a letter, concentrate on the fact it is unfortunate you have fallen out and not what you wanted.
In my humble opinion you will probably not be able to agree about all this and she will not see it from your point of view. It may be that she is jealous of you, you have the new baby and all the attention at the moment.
It's just my opinion but if the children are a problem between you, in that her son behaves badly with your daughter, I would concentrate on just being friends with her and see if you can go on that way for a while. Chat on the phone, email or whatever and just go out to lunch sometimes and not doing the family stuff with kids?
Just an idea.
But remember, you are right, if her son's behaviour risks your dd or new baby in any way, you owe it to them to change the situation, and if he is leaving inappropriate messages, you owe it to your dd, as you are doing to stop it.
I am just curious that you say she is your mum's favourite. Is that definite? It almost sounds like she doesn't feel so secure of her place in the family, because if she is the favourite and knows it, why make your life a misery. unless she is just a not very nice person?
Can you talk to your mum about things in general?
Congratulations on your new dd.
I hope you find a way ahead.