The baby/ toddler stage is just another form of active rhyme time play group.
The advantage that it had for me was that I could take two DCs swimming safely on my own when it would have been too logistically difficult to go solo. DS1 could sit on the side by the instructor with a toy car while I was in with DS2. DS2 could be in the car seat/ play pen while I was in with DS1. It was also (light) exercise for me at a time that any small wins were worth claiming.
Also with DS1 turning out to be dyspraxic and having sensory issues, it was always just something he was used to and a good range of whole body movement to do regularly. He could have been difficult to introduce/ re-introduce to later. (With my issues with swimming/ cycling, dyspraxia in the family is no shocker)
The infant school years are probably when there's least pay off in benefits/ learning.
But for us with a close age gap I was there with one of them anyway and the lessons normally worked together well so they both had lessons on the one journey.
I wouldn't rush to recommend staged lessons for 4-6 year olds in terms of learning efficiency for someone who wasn't already doing it. 7 is a more receptive age for learning effectively.
Our progress was delayed by dyspraxia, a phase where my work committments interfered for a while (so he missed lessons every few weeks for a few months) and a year written off by Covid. Aside from the enforced months of closure, DS was ready to move up, but jammed by avaliability and a freeze on progressions because a lot of children regressed March- Sept so the previous gradings weren't necessarily accurate.
Ultimately we've come out of it with a strong, competant swimmer, and a lot of regular exercise on the way. For me that's a fair outcome and we could have stopped years ago if we wanted to. I wasn't bothered about getting to that outcome ASAP.