Its not the same situation, but we're in middle of a party wall award process with our (unattached side) neighbours. They are building a basement, and its within 3 metres of the party wall (garden wall, in this case) so they need a party wall agreement: we're in midst selling process, so we said 'not without a party wall award'.
AFAIK,you can't actually just say 'no' to building - unless they are breaking planning permission rules - but you can say 'not without a party wall surveyor of my choice'. The building party has to pay the surveyor's fees. What happens then is that both parties appoint their surveyor (or just the one for both, if they agree to, which saves on costs) and the surveyor/s assess the planned work, photograph and check your property, and eventually, usually, award a party wall agreement. They come back and check your property after the work is completed, and make sure there has been no damage to it as a result of the work.
TBH, even if we weren't mid-sale, I would have asked for a party wall surveyor to get involved - its good protection!