My children are all far, far older than yours, OP.
However, FWIW:
Our DC went to a local prep school until they were 13. We just loved the prep school, and I can't believe there's any child on the planet who wouldn't have loved to be there. It was also a very short walk from our house, so it was convenient. Having looked at that school, we didn't bother looking anywhere else.
We chose their next schools based on their interests/abilities/gut feeling.
We are very much based in the place where the DC went to prep school, so couldn't go around sticking pins on a map and having geographical debates. I think that, if your children are under five, you really won't know at this stage whether boarding is a good idea or not. It's fantastic for some, and less so for others. My youngest is about to board, at her insistence, having seen her brothers do it.
I think she'll love it. If I didn't think that, though, I would say no. Her current school goes up to Upper 6, so she could equally well stay there.
Having had children at boarding school, I think it's absolutely brilliant. However, my family is used to boarding (XH and my dad were both boarders), so I don't know if that plays a role.
I do know, though, that I would not send any child to board unless I thought they would relish the experience.
It's more about the particular child, and less about the type of school - if you have the financial luxury of choice.