I know it's not a race to the bottom and I truly appreciate how difficult you're finding other people having more than one but you really do need to sit back and take stock of your own luck, the way you have done with other people's. Rather than hyper focusing on not having two, celebrate having one at all.
My parents could've conceive naturally and waited years to adopt, I know my mum found it devastating not to fall and still finds it really tough seeing people who are pregnant as she wishes she had been. I know she will likely find it tough if I fall pregnant (we are trying) but she explained to me that her sadness she missed out on an experience doesn't mean she will be any less happy for us.
When you say your son went to university, it will have been painful for people who had and lost a child, for people who never had children but wanted them or for people whose children were unable to go to uni due to health issues for example. Have you thought about that before every single conversation you've had with anyone about your son going to uni?
When he was little were you careful never to mention his milestones in case other people had potentially had and lost a child, had never fallen etc etc? Of course not - because it would have been an unreasonable expectation to be hyper sensitive to every possible trigger anyone else could experience.
Everything we have is something someone else wishes they could have. I'm finding it hard seeing what feels like everyone I know getting pregnant while I'm not yet, but I don't think they should have to hide their own joy - because I will be full of joy if it happens for us too.
It's tough but you have to try not to let your sadness for yourself outweigh your ability to allow others to be happy, without you resenting them for it.