I just messed around on a keyboard from age 2. My dad is often telling anyone who'll listen me that he heard me playing Ravel's Bolero as I'd heard it on the ice skating :o
I didn't have formal lessons until I was 11 but I was fairly good by then and started at grade 5. But I was lucky that my mum was quite good and was able to teach me notation etc - she had never had lessons, neither did my grandma.
It certainly did me no harm having no proper lessons at a young age - maybe it prevented the boredom/burnout that some children suffer? I guess it would've been more tricky if I didn't have a musical family though.
I ended up teaching at my music school and in 5 years we only had one 3yo pupil. I think it's too young unless they've already shown they have an amazing talent for it TBH.
Do you know anyone good at a particular instrument who can just show them how to play informally?