Start with the Secondary Schools. It's been A LONG time since I was there but you have Elgin Academy (DID have an academic reputation, music), Elgin High (DID have a reputation for great sport, DoE that kind of thing).
I have a feeling they MIGHT have merged now - someone will know better - but your kids obviously easily need to get to school somehow.
I HEARD Milnes had / was getting a better reputation. Lots if kids bussed there from Elgin. If you can afford it, Gordonston is local.
West End PS, Bishopmill PS WERE the posher ends of town. Also around St Sylvesters PC. No idea if that is still the case.
Where are you now? What facilities are you used to?
As pp, the town centre is VERY sad. Lossiemouth is lovely but you will be a taxi into Elgin for sports, swimming etc. If you want to shop in M&S and nip to Pret for lunch - forgot it. It still has a cinema I believe and I think the ice rink is still open.
Talking to friends who are still there, the nightlife is crap. Some good restaurants though.
The 8 Acres Hotel used to have a pool you can join as a private member. Think David Lloyds and then lower your expectations massively.
Moray, in general, I've found a lot friendlier than Aberdeenshire towns, for example to outsiders.
Look around Dr Greys, Bishopmill, near the station but not the "New Elgin" side of the tracks. There has been a HEAP of new house building so can't comment on those areas. Check local flood maps. Areas around the Cathedral are prone to flooding.
Winters are not too bad - unless you live rurally. Fochabers has a bypass for a reason - there's nothing there.
Aim for central Elgin so at least your kids can get places under their own stream.