Exactly this.
When you celebrate your baby's first birthday, it's a milestone for all of you. A year ago you were at the hospital just meeting your child for the very first time, and if it's your first child, becoming parents. Now you're celebrating a whole year of getting to know your child, surviving the challenges that the last year has thrown at you, the first year of your new family in its current form.
It's weird for a parent not to want to be there for that.
My parents have been there for most, or possibly even all of my children's birthdays, and they live in a different country and are not my children's parents.
I would find it so strange and sad if my husband didn't want to be there, taking a full part in the day.
My son's first birthday was on a Friday and my husband and I both had to work that day. His childminder made him a little cake with a candle in it and sent me a picture of him with it in his high chair at lunchtime. My gut reaction was sadness that she celebrated that milestone with him first, and not me. But I got over it because he didn't know it was his birthday, she was doing a kind thing, and we celebrated with him that evening and at the weekend. I can't imagine what is going through the head of someone who opts out of a big chunk of their child's first birthday in favour of going to the bloody gym! Like, if you really must go to the gym every day including on your child's first birthday, just go for an hour when they are having their nap. Don't make a half day of it and leave your partner to do all the birthday stuff as well as the normal childcare stuff on her own.
If my husband did this on our child's first birthday I'd probably take it as a sign that he didn't think our child and our family was that important, and that over the years many other events and milestones would play second fiddle to whatever he wanted to do. I'd be anticipating 18 years of going to school plays and parents evenings on my own, and being the one to take my child to their hobbies every week because he would always have something more important to be doing.
Because if you can't be arsed to make an effort for your child's first birthday, it doesn't bode well for their second, fifth, tenth or 18th. Or any of the days in between.