YANBU, the pool is for everyone to use and you can use the 'free' bit how you like.
I'd simply reply with "Excuse me but could you get out of my way."
Any further argument and I would say 'lengths lane swimming is over there."
Any fights, report to life guard.
I;d be tempted to say "Please don't be rude in front of my son." as well.
You could also report to life guard if it happens a lot. Surely teaching children to swim is one of the main reasons for public pools!
(Or in reality I might mumble sorry but really I want to say what I said above!)
"WipsGlitter* Re "Could you drag him up and down rather than across?" Unless she is pretty tall she would not be able to stand up in the deep end and non-swimmers (rightly) cannot go past a certain depth at some pools.
DeathStare if 'Free use is free use' then presumably walking backwards is free use too! People swim back stroke which means going backwards. but there is a separate but for lanes so swimming lengths in the free use park means the swimmer is being a bit selfish unless the lanes were really crowded. And they have no right to be rude to others doing 'free use'.
MargotsDevil 20 months isn't a tiny bay and lots of babies start swimming from 5 months, mine did.
The Op has clearly stated he cannot swim in the toddler pool. No one can, it is for paddling in. He is having swimming lessons. I doubt he knows he is having a swimming lesson, but he is. How can anyone object to a toddler learning to swim!
OP re "I didn't mean to come off as entitled" you ABSOLUTELY do not come off as entitled. Maybe some people do not know about baby swimming but whether they do or not a random person being rude to you in the pool is very nice. And you were totally in the right.