I always took them swimming from a very young age, but the oldest to start proper lessons did so at age 5, the younger two started at age 3.
We did have a couple of years where they could only swim in lessons though, as DH couldn't come with us and we couldn't meet the pools required adult/child ratios with 1 of me and 3 of them.