Legally, it seems clear that your brother has no legal claim. HOWEVER, I understand your moral dilemma. Had the deaths of your DM and DGM been reversed, with your DGM dying a few months earlier prior to your mum, your father would have inherited; the wealth would have passed from DGM to DM, then to DF. As DF is still alive, your brother could have reasonably expect to inherit 50% on his death, in the absence of any wills to the contrary. This isn’t what happened though…
I suppose your brother anticipated that he would receive 50% of your father’s estate on his death and nobody considered what the outcome would be if your DM pre-deceased your DGM. Yet the maternal family wealth was discussed (allegedly) and possibly expected to filter down to both of you? But, possibly not. Your parent’s wishes could have changed dramatically after 20 years of no contact.
Certainly your mum expressed her wish to you that you should inherit the family wealth alone. When and what wishes were expressed to your brother is uncertain, despite what he claims now. Unfortunately, nobody recorded their actual intentions in a will and you are left picking up the pieces.
His mother’s inheritance is irrelevant, as his mother and father were not together at the time of her death (unless they were still legally married?) It is different to your current position, because your father (who you might inherit from) did not inherit SB’s mother’s estate.
Give him half if you wish, but don’t expect this to preserve your relationship.
You are vulnerable now, having suffered significant losses in recent months. My advice would be to make no decisions at this point and allow yourself time to grieve. If the sums are substantial, it will take some time for the estate to be administered. Delay your decision and any further communication with your brother on this subject until you are ready. Arrive at a decision that you are comfortable with, independently from any pressure from your brother. Tell him to back right off. Also, advise him to take legal advice in the meantime - he will have to acknowledge that if you decide to give him anything at all, it will be out of love, not legal obligation. I’m sure if you have a strong, true relationship, he will understand your need to grieve before resolving financial matters. If he is impatient, I would question his motives.