Give him lots of opportunities to play music, listen to good music whichever genre you prefer.
Open his mind to all different sounds.
Go to a music shop and buy some small percussion instruments.
I got a huge trunk, like pirate chest full for under £100, but was about 10 years ago prices 
Failing that, make your own cheaper versions.
Save the lessons until he is older, he will probably gain much more.
Music shouldn't be a chore, or hard, but fun.
Singing is free and is so good for their soul 
My dd singing from months old, then echoing bird song, transport, usual everyday sounds. We showed her bits of music and she started formal lessons about 6, on violin. Teacher was recommended by a friend, we trialled for a while, it didn't work so we changed teachers and it was a good relationship for a few years.
I think it's important to let the child find the instrument. It might be completely different to what you'd choose. Watch some of The Proms with him and point out the different instruments. Listen to Peter and the wolf. 
Some instruments you can't play at 4, this was why we put her off until she was a bit older, she still chose violin but she could have changed her mind.