I hope you gave them some advance notice, as once you deliver the party wall notice, the clock starts ticking for them. It's not always convenient for people who are new to party wall stuff to have the time to understand all the implications within 14 days, on top of whatever else is going on in their life. Our neighbours did this, no warning - just put it through our letterbox without even knocking, no cover letter either. It put our backs up as we were dealing with a lot of life crap at the time and then had this time-sensitive complex topic to learn about and deal with on top. If they'd come to chat to us beforehand about what they were planning, which they'd known about for months, it would have all been a lot more amicable. We'd had a good (but distant) relationship with them beforehand, but how you deliver the news over impending noisy building work lasting months makes a huge difference to how smoothly things go, and may account for a negative response.
Also they just printed a notice out off the Internet as that's what their builder told them to do. It should have been a straightforward notice from their perspective, but they still made at least three not immediately obvious mistakes with it, each of which would have rendered it invalid on their own. We found our own surveyor who told them this as they didn't believe us initially, and they had to start it all again, only doing it properly this time with a surveyor of their own. This is why it's so important to get somebody to do it properly! And yes, it's quite reasonable of them to get their own surveyor (which you have to pay for) - it's generally recommended. Our neighbours very much held that against us, and by their own admittance were deliberately spiteful to us as a result of it increasing their costs fractionally.
To answer your question, there's pretty much sod all they can do to object - that's to do with the planning permission (if it's needed) side of things. If you haven't got planning permission, be absolutely certain you don't need it and check for yourself, because if they are going to object, and it turns out you did need it and they discover that, it could get messy. Party wall is just to protect their property, and there's no realistically enforceable clauses that can go in it (sadly, even stuff about the volume of builders radios or blocking driveways isn't worth putting in party wall agreements as it gets ignored and nobody enforces it) that would interfere with your building plans.