Ahhh, you're sort of spread across the state then. You'll both have a fair commute even meeting in the middle, I'm afraid, as the two are pretty far apart! Are you working north of Amherst or south? If north, you'll be wanting somewhere on Route 2 (like Harvard!) but if south, you'll want somewhere on the Mass Pike - maybe Westborough would be worth looking at? It's quite nice and has decent schools.
One thing to watch out for when looking at schools is if your children will be in high school at any point, many of the high schools are regional schools covering several towns - e.g. Nashoba Regional High, Wachusett Regional High etc, so you can be in a town with a very good elementary school but which then feeds into a mediocre high school.
Harvard is great! It is fairly expensive for houses but I would personally buy a smaller house there as opposed to a larger one in another town just to get into the school system. There's information on the schools here: www.psharvard.org/ The school is one of the best in the country and has quite a few Ivy and little Ivy acceptances every year. They are very strong in drama, music and sports as well.
The town has a pond and in the summer everyone gathers there- you can take swimming, sailing, canoeing and kayaking lessons. The general store is under new management and holds Friday evening barbecues in the summer. There's lots of fairs and festivals on the Common and high school grounds. There's tons of conservation land for long walks and cross country skiing in the winter. You can do downhill skiing at nearby hills or it's not a long drive to NH (also for tax free shopping, Nashua is not too far away!). It's a small town but it has a GREAT local community! It's apple country so lots of apple picking in the autumn. There's also quite a few horse farms if you're into riding.
Honestly, I can't say enough good things about Harvard! (If you do take a drive through at any point, go past 40 Slough Rd and give my childhood house a wave!)