I feel this falls into the category of ‘pick your battles’. Mine is 3 and stopped recently of his own accord. Tbh he’s only had water in a bottle, so I felt it wasn’t an issue requiring intervention!
if it’s milk, slowly switch to water. When they’re ready, choose a grown up bottle/cup together. In the meantime, keep offering a cup or bottle. Let them decide.
don’t sweat the small stuff.