Our primary set for maths from Y3, we have a big intake/primary with several parallel classes.
This is ok to my mind as long as the sets are fluid and flexible. As some may know I have some strong feelings around 'ability'! 
Head has an open door policy and is very friendly. We were talking about setting and he said to be honest children don't tend to move after they've been set, it is just to difficult to keep moving everyone around. Of course, if a child is showing exceptional promise etc and working way beyond where they've been placed they'd be moved.
Makes me think about how rare it is that a 'bright' child is re-categorised after a run of poor results (other reasons are found, falling in with a bad crowd etc) compared to the other way around.
Also think about the dreadful teachers that they gave the bottom maths sets in my day..but it's not like that now? She says, brightly
.
Would this bother you?