When my daughter was born I stopped doing Mother's Day with my own mum on the actual day. Unfortunately there is a history of my mum (or dad) getting too drunk at family events, or they don't like the venue, or the food isn't up to their standard. There is always some comment about the service or one dish being too salty. There wasn't enough effort/money spent on the gifts. You get the idea.
Invariably I would always come away from the day feeling down/annoyed/embarrassed by drunken behaviour.
I didn't want to feel like that on Mother's Day when I became a mum so the day is just for me. DH understands the dynamics and is happy to support me. He sees his mum in the run up to Mother's Day (she was not overly fussed by it when they were growing up anyway) but we always do something just us.
I always arrange something with my mum in the run up, like an afternoon tea or a lunch but never on the actual day. She had 35 years of having the day for herself and now it's my turn and I want to enjoy the day with my DC.
As for Father's Day- there is a big yearly sporting event that DH goes to every year and it always falls on Father's Day so he just gets a card and a present packed into his bag! Sometimes I'll see my dad on the day/do something with him but often they are on holiday over Father's Day as well.
I think it really depends on your family dynamics and the relationship with your parents and in-laws about how you celebrate it. I am all for doing your own thing on the day once you become a parent- as I've said, your own parents have already had 'their time' when you were small, so there's no harm in wanting to make the day about you now you are the parent