YANBU to do lessons at 5. There is a wide range of abilities and personalities that affect how well lessons go, but just on age, it isn't too young. It also isn't automatically "late" either - there are some limits on a few things without it, but that's not the same thing.
DS1 was in the pool young, did the ducklings thing and had lessons from about 4-5. He's 16 now, hasn't swum in years, not sure of his current ability as he has no interest in it but he says he's 'not much of a swimmer'.
DD1 was in the pool young, did the ducklings thing, and decided at about 4-5 that any water she could be submerged in was trying to possess her, would cling onto the nearest person with surprisingly strong limbs and continued to be that way for years. Bath times were fun /s. It took her being 12 and getting a leaflet from the school that said only students who could swim could go on the cool trip before her father and I could teach her how to not freak out in the water, float a bit, and be confident to swim on her back to the side from the middle of our local family pool with it's "non swimmer' section. She started to enjoy it to the point she was saying it's her favourite sporty thing when pools were closed last year. We'll get back to it when we can.
DD2 was in the pool young once, when her skin reacted badly (she's had eczema since less than a week old, in the pool I could literally see the reaction) and we were advised to wait longer. Then she started to get skin infections all the time and we were advised not to take her into a pool, the sea/anything similar because of the raised risks, she had to have medicine in her baths and had to wear coveralls pretty much 24/7 because at times it could over 2/3rds of her. This obviously made it harder for her siblings to swim too. She was about 9 the next time she was in a pool, but she has to lotion well before and rinse and re-lotion well just afterwards. She can doggy paddle and dive down happily, do a bit of floating but isn't much for staying still.
I used to feel guilty as others describe, that I must have done something wrong for DD1 to be so upset and how was I going to keep them all safe when I got some very good advice - swimming is just one of many life skills, water safety can and should be taught separate from swimming & knowing our limitations and what we can and can't do about them & making choices on that are important life skills too and an important part of water safety when we have way too many young people getting hurt and worse from jumping in the reservoir or the river overconfident that their ability to swim will keep them safe.