My parents took me and my sister out of school quite a few times, we didn't grow up in England though and it wasn't frowned upon .
We were almost straight A students, got our degrees , postgraduate degrees and good jobs. I don't remember it being a big deal.
I think if it's your last year at school or pre GCSE's exams then it's probably not a good idea. But at primary school i really don't see what the problem is. Although all kids are different, my niece for example really panics when she has be miss school even when she's ill and has doctors note. She hates missing school and she's only 8.
It's really parents' and kids' call, depends on the circumstances,