PP raise very good points. The villages around Frome are nothing like Frome itself wrt character.
Public transport is next to (or with recent cuts completely) non existent in many of them and it can feel very isolated in winter. With two small children you will end up driving everywhere until they’re teens and have their own cars - implications of this for schools runs, activities, friends, doctors etc.
It sounds daft to say it but no takeaways, no Uber (I think Frome has about 5 taxis), some of the pubs in smaller villages have really limited opening hours. You also have some massive quarries in the area so probably want to consider proximity to those for blasting.
OTOH it can be idyllic when the weather is good and as much as I enjoy outdoors activities, I value not having to drive absolutely everywhere.
Some bigger places I’d aim for:
Bath (sheds of shite and tourist hoardes aside)
Chew Magna
Wells
BOA
Frome itself
Langford
Definitely try renting first and see how you get on.