I'm sorry, but you are the one cherry picking reasons not to take him to school and you have no excuse for not ringing the school out of courtesy to say you will be absent.
Ringing up in February to apologise for missing an apt at Christmas? really?
Unless you have a 1920's silver X, prams and buggies these days are so light they can be lifted with one hand. I don't buy into being unable to get down the stairs if the weather is bad.
You must have agreed to outside agencies coming to assess your child. These notifications and requests are done in writing.
These sorts of thing are two way - if you really want to move and improve his attendance, you need the school to buy into this and back your application for new housing. If you persist in ostracising them, they won't help you.