I agree with what's been said already. Your DS is roughly the same age my DD was when I split from my ex. He moved out 4 months after we split into a 1 bed room flat, but DD stayed overnight fairly early on. She has a lot of affection even now for that flat, as whenever we pass it, she always comments on the fact her dad used to live there, and that she wished he still did. The transition is important but it's not difficult especially at such a young age. Even though my DD has strong memories of her dad's 1st flat, she has no memory of him living with us despite me not having moved since she came along. I used to spend a lot of time at her dad's new place to help her settle there, and didn't leave her until she was completely happy and comfortable. It took a good few visits lasting a few hours at a time before she looked at ease and able to stay without anxiety. It was never an issue of her being worried about staying with her dad as she was used to him and comfortable with him. It was just a new environment, a slightly different set up and she had a little bit of anxiety about leaving me. All I had to do was make sure she could see how relaxed I was with the set up, and how happy I was for her to stay i.e. make sure I was excited for her etc.
Tbh, if you and your ex have a good friendship, and are able to communicate easily, then you will have no problems at all. You clearly have confidence in your ex's abilities to care for your child as you do, and that is a huge plus in these situations. All you have to do, is make sure that you both seem happy and positive about the new home, make it something exciting and positive for your DS, and be there for him to help him ease through the transition. He probably won't notice your ex not living there (much) and when he does, you just ask him if he wants to speak to his dad, phone your ex, and let them talk about when he's next going round there. Or take him round if that's possible. Just being able to let him have some form of contact with his dad when he ask about him, or seems to miss him I think can be really helpful. I was lucky in that my DD didn't ask for her dad at all, although she loves him, and would happily go and see him when she could.
I think sometimes the anxieties some children have in these situations are when there are barriers in place that don't help alleviate anxieties that any child would have about a change they may not be too sure about. As long as you and your ex are able to maintain an amicable relationship with good open communication and flexibility, then your DS will certainly benefit hugely from having 2 parents who are both focused on doing their best for him.
Good luck, hope it all goes well for you all.