thought I'd add my opinion from a child's POV, though granted this was in the late 80's/early 90's. My Dad was posted to Lagos when I was 3 and my brother 1 month, we were there for almost 4years in which time I went to school at St Saviour's, which is I think an International School but had a lot of Nigerian children there too.
Of course I'm looking back through rose tinted specs but I had an incredibly happy childhood there: beaches at the Creek, swimming every day at the pool at our compound or the British High Comission, the great thing about compounds is we played outside with other children all the time, which isn't something I'd let my children do here in London so for child it feels very freeing. Because my Dad worked for the British Council we had a driver and a nanny, our nanny was amazing and I'd often to go to her house in Lagos and got to visit her village a few times too. I know my parents had issues with their first driver trying to steal from them but the next guy they got was lovely.
We had a minibus organsied to take us to school which if I dredge my memmories enough yes did have armed guards but I certainly didn't grow up feeling in danger or unsafe (not to mean it isn't dangerous of unsafe but there are definately ways of making children feel happy there).
The BRILLIANT thing about being posted anywhere in Africa is the opportunity to then travel round other parts of Africa, and again I have tons of memories of Ghana, Benin, Gambia.
When we came home I remember being utterly perplexed at the first encounters of racsism, I just hadn;t come across it despiote the obvious divide between expats and 'help'. Living there fostered a sense in me to travel much more, to live in a vibrant and varied area and my childhood memories are of sunshine and swimming.
Just thought it was worth mentioning seeing as you have two children.