i think it depends how much time you spend on it..colours are something like numbers which i would always use when speaking to children from the start
so I always refer to the colour of something in conversation
also play rhymes and things with colours