Firstly, phone the builder and ask if he uncovered any asbestos in the build. Assuming you trust him, he is unlikely to lie to you, because a) if he had, and he concealed the fact during the works, he will have broken the law, and b) if he had, and you did not know it was there beforehand, he would have been entitled to be be paid for both his time and labour over and above the cost of the job to deal with it. Few builder will walk away from that extra couple of grand when you are on the hook to pay with no dispute.
So rest assured, the odds of asbestos being preset are remarkably slim.
Secondly, asbestos related respiratory problems are overly feared and misunderstood. You really need prolonged and ongoing exposure to asbestos to suffer harm. So digging around on the internet should assuage your concerns.
Thirdly, most councils know exactly what properties have asbestos in them, because it was identified as a product in the original building warrant application back in the day, and if not that, then someone in a similar property extending at a previous time would have uncovered it in their property and the council will have flagged all properties in your neighbourhood as being candidates for future concern. Especially if you have old council housing stock where they know exactly what materials were used. It would have been flagged up when you made your original application to extend your house.
Lastly, it sounds as though you have a dusty house that is likely a combination of fine particles of sawdust and cement (plasterboard). Nothing a good clean all over with the bristle brush on your Dyson shouldn't solve. It's something they should have been considerate enough to do for you, but hey ho. I don't know what you asked for in the first place.
Aside from needing to clean and it taking more than a once over with the Hoover, I wouldn't be too worried.