Pretty sure your Ex can't just take himself off the mortgage. It's a debt he signed up to and he would need to pay the mortgage company their money back or have someone else take over the debt in order to be relieved of it. So if the Mortgage company aren't prepared to have you take it over and no one will buy it, your Ex can't come off the mortgage and it will continue along on the terms it was set up on in his name.
Assuming you are going to be splitting any equity, it would probably be best to talk with your Ex about whether to lower the price you're prepared to accept to try and secure a sale.
The mortgage company can, in some circumstances, call in their debt, which if you can't sell yourself or pay them back would mean repossession and then they sell (often at a much reduced price with you getting little back or even, potentially, still owing them money if the sale is for less than the mortgage amount). I suppose that's a horror story "Worst case" scenario. But I've never heard of that happening in a break up situation where the mortgage is being paid.
I think your most likely worst case is that Ex gets a court order requiring you to sell by X date then, if you fail to meet the date, he might be given authority to be the sole approver of any sale. It would still have to sell, though. Which seems to be the tricky bit. It's not that you're dragging your feet which is what an order like that would really be useful for.
If you were still a couple and trying to sell he'd be stuck because of the problem with selling, and he's stuck now. Your Ex doesn't get to simply duck out of that difficulty because he left.
Having said all that, I'm not a lawyer and you should probably talk to the lawyer handling your divorce to see what a court might actually order.