I think you need to do what's right for your own children, not listen to others. Having said that it's worth checking if the local comp has a grammar stream.
We're in an 11 plus county and DS sat it - his teachers said he was definitely good enough and he was in top 10% of his class in spelling, Maths and English in year 4 and 5 tests. He had a reading age of 12 when he was 6 years old. Not typing that as a boast, just a link into the next bit.
He passed his 11 plus with flying colours. But, in between we'd been to see the grammar school, which he loved. Then we went to see the local comp and he completely changed his mind. The options on offer are huge because they have to cater for the academically bright, the sporty, the less academically bright, the more vocational kids so everyone has an opportunity to thrive.
He had to sit a mini entrance exam which 'streamed' the kids. He's in the grammar stream (not surprise really). Many of his friends who were 11 plus passers have gone to the grammar schools but a few have gone with him to the comprehensive. They all love their schools, all seem to be doing well and that's the main thing.
I went to a streamed comprehensive - the fact that kids can move up a level is great. My best friend did exactly that, from middle band to grammar band. Kids mature at different ages and an average academic kid at 10 can be a real achiever at 13. I'm already seeing the difference in DS from primary to secondary school.
Worth checking with the school if they do stream the kids then it'd probably be the best of both worlds if the 11 plus just seems a bit too much. Don't forget too that some of the 11 plus kids won't be offered a place. It's not just the passing of the exam but the points they get. Grammar schools only have a fixed number of spaces so if 500 kids pass and there are only 300 spaces the lowest 200 may not get in. So some friends will be in the comprehensive as a second choice anyway.