Builder here
Your estate agent is talking from his bottom, or trying to rip you off. Asbestos was banned from use in new builds in 1999, prior to that its quite common, particularly for soffit boards, which I suspect is what it is in your house.
Asbestos is actually a mineral, and is pretty much indestructible, it can't rot and doesn't burn, the asbestos soffits on my 1996 house will see me out, the only issue is re-painting which is a pain and where plastic wins.
Unless a house has been stripped of its asbestos anything constructed between the 1930s and 1999 is guaranteed to have some in it.
In Canada it was only banned in 2018 and was in common use before that, it is still used in much of the world, including China, that produced 22 million tonnes of asbestos products last year.
To be clear, it can be dangerous if disturbed, but is not an issue left alone, and should not impact the sale of a house.