Baggy jumper sleeves

[CantChoose]

I embraced cosy jumpers last year having been in a job with no long sleeves allowed before then.
I've tried to follow advice on here about not washing them too often to keep them in good condition but I find the sleeve ends go baggy after just a couple of wears and look awful.
I have a tendency to roll the sleeves up above my elbow as I'm still getting used to sleeves but it seems to happen even if I'm careful not to.
Any hints?