YANBU.
I think for your happiness and peace of mind, you do need to have a full and frank conversation with your DH, setting out how you feel. Or maybe write it in a letter if things are getting heated between you.
Something like...
"I am hurt and confused by (the other thing between you).
I feel like it is getting in the way of us talking sensibly about birthday lunch.
So I am going to try to talk sensibly about birthday lunch.
I am tired and down and need to recharge. Immersing myself in the weekend long event will do that. And just now I really need to do that. The clash of dates is something I wish could be avoided, and I have tried very hard to avoid a clash, but, as is unfortunately often the case, I am the only one trying to do that, and I can't do it on my own.
It is unfortunate that this will lead to repurcussions from ILs and I am sad that that will be a consequence. I wish it could be otherwise, but that behaviour isn't caused by me.
You are very good about how often we see my family, and I really appreciate you doing that. I hope that you see that the amount of time we spend with them is part down to your good attitude and in part down to their good and flexible attitudes. I wish I could build as good a situation as that with your family, but it is difficult to do that when I am the only one trying.
So please understand why I have to go to my event. It is partly because I need to recharge and it is partly because I cannot build a good situation with your family when I am the only one trying.
I do thing we do also need to address the other thing and I think to be in the right place to do that, I need to be able to take some time for me at this event. I don't do this very often, so I hope that you can understand that I only do it when I really need to."