Pruni, dd has moved schools 3 times. She went to nursery at 3, loved it and made a close friend there, then we buggered off to France and she started a new school and made another close friend, then we came back and she is at yet another new school. It won't last as this house is rented so at some point we'll have to move again. But I have found that short-term it was difficult and hearbreaking, but long-term it has boosted her confidence, she is more of an individual now and makes friends easily.
If you give children these challenges to face early on, they deal with it much better and it actually helps them when they become older and have problems. If a child grows up in a lovely, secure, stable environment with no change, how do they deal with the realities of life later on? Your ds will benefit in the long-term, but in the short-term it will be tough for him, and you.