I don't see why so many people are advocating sarcastic responses or passive-aggressive replies or getting so het up about being asked.
I don't think it was unreasonable in the least for them to have asked. They didn't demand, they didn't expect, they just asked. You can say no, but no reason not to say no nicely, especially given that you have to live next to them for the foreseeable future.
But I would try to say yes. It could be difficult and expensive for them to try to get some kind of office space for what might just be an infrequent but very important call, and your builders might easily be able to find other jobs that need doing in that time that doesn't actually affect you at all. So I can totally see why they asked if it might be possible. Why not ask the builders if they can accommodate it, and if needed, ask your neighbours for a more precise time?
If they are otherwise good neighbours, if there's no indication that this is the start of many requests, then why not just try to be helpful if you can? You might need them for something in the future as well!
No need to assume the worst about them though, and if you have no other reason to think that they are going to mess you around, just politely chat and see if it is or isn't possible - but don't have a go at them for just asking.