Fishguard is in a beautiful setting, like all towns some parts are much nicer than others.
If you were there today when the sun is shining and the weather glorious you would probably move in a heartbeat. Remember that for most of the year it is grey or wet and windy.
The primary school is just coming out of special measures, so that might be an issue for you. They have an English stream and a welsh.
There is also catholic primary school (English medium) in fishguard and a primary school in Goodwick. The Goodwick school will have a new HT is January.
The secondary school has a HT who has been in place a year. Rumour had it that she was going to make sweeping changes. So far, there hasn't been much evidence of this. A lot of staff are leaving at the end of this term.
Bear in mind that unlike larger towns there isn't really a choice of secondary options. Most children just go to the nearest one in their town.
Employment can be difficult to find in the area - whould you be able to find work without having to travel?
Public transport isn't great.
If your children are into shopping and city type activities they'll find it difficult. If they are outdoor types there are lots of things they can do like Sea Cadets or sailing. Because of the rural nature of pembs be prepared to spend your time doing a lot of ferrying them around.
The town centre is small, there is a Boots, CoOp, Peacocks, small health food shop, optician, deli, florist, butcher, green grocer, wool shop, cafes and some other small independent shops
Down in Goodwick there is a Tesco Express.
If you are looking for a hippy vibe in pembs I'd suggest the Narberth area, although it isn't coastal and house prices will be higher.
Newport is lovely and may be more your type of town, but is lots more expensive.