So they've buggered off. How rude. What do you plan to do next? Idk if I'd want to allow these oafs back on your land. They sound like cowboys. Perhaps speaking to a solicitor would be useful and you could ask another contractor to do it and charge ndn but this would likely lead to small claims..... and obvious disputes. Also, would any contractor need to use the rubble to reinstate the wall? Ie is it a specific material? What a mess.
And definitely need to inform the council. Also can you report the company to trading standards? How disgusting to modify a listed building.