YANBU. When the kids are that small, and there're two little ones to get fed and to bed, one of whom is breastfed, 3 nights a week at the gym, plus other activities is too much.
Your DH is being selfish and not considerate of you OP.
My kids are similar age. DH gets in (straight) from work just after 7. 5pm - 7pm is really hard: clearing away tea things, tidying up house, the big one wants stories, time and attention, little one wants boob, both are tired and grumpy (fights over teeth-cleaning, hair-brushing etc) practicalities of getting them both changed, bathed etc. And if you're up in the night and breastfeeding it's a real tired time for you as well.
In these circumstances, DH coming home at 6pm instead of 7pm makes a massive difference to the whole family.
In general, the MN rule seems to be that both people get roughly equal time to do their own thing, and that if this would mean not enough couple / family time left, then the individual time needs to be reduced, but still equal. In this case, DH is taking the time for himself and you are run ragged.
Also, if one of you is totally knackered while the other is scooting off to the gym that often, then there is an imbalance of work going on. Take a look at your DH and in the mirror, and there's your answer.