I'm not sure if they woudl allow this . There have been problems in the past when an adoptive father who abused his child turned out to have abused his first wife .
Other times it turns out that someone has a child by their first marriage who they don't keep inconstant with or pay maintenance for .
You will understand why these things raise big concerns in adoption. And most people in these situations simply say " oh don't contact her , she's crazy / an addict / mentally ill" .
So the usual practice is for them to interview former wives or long term partners .
I undertstand that it's upsetting for your H, but he wouldn't be contacting her himself , it woudl be SS. And if he's unwilling to deal with painful issues from his own past, SS may feel he's not very well equipped to help a child deal with their own past ( which is probably much more upsetting that an 18 month marriage that didn't work out ).
If your H is still so upset about it , Perhaps he might find it helpful to go for some couselling now, as once a child is placed with you it may bring up distressing issues for him . Also if he is still too upset about it it might not be the best time for you to consider adoption . There's nothing wrong inputting it on hold while you work through these things .
SS are used to the fact that some former partners are bitter and may say negative things about their ex. She is not being asked to give her approval for your adoption .
I hope you can see they have good reasons for this request, it's not just nosiness, it's about a Childs safety and welfare .