I'm sorry for you and would like to put the other side of it, if I may.
I had a child with a married man, years and years ago.
Contact was initially secret but once his wife knew, she would call me to threaten me with taking my child away, saying it was conceived during their marriage and therefore belonged to them.
I could understand her anger but you need to be very clear about what is best for the child, and make sure that your inevitable, and completely understandable and valid anger about all of it does not in some way come out against the child.
I was all for letting them see my child but my mother stepped in and told me not to be so ridiculous, as the wife would undoubtedly hate the child because of its provenance, and that although she may not be aware of this she would somehow show her resentment in a subconscious fashion.
My mother may have been right or wrong, we never found out as I only allowed him to see the child on his own, without her present, and shortly afterwards the contact stopped. I have a feeling he could not handle being in that position.
My child grew up without having any contact with his father. It is sad but at the same time not tragic - he is a lovely person, rarely mentions his father, and I don't think he is too traumatised for not seeing him.
Just be very careful that you don't want this contact yourself for ulterior purposes - ie to claim your rights over the child's mother, or demonstrate ownership etc as this may damage the child irreparably.
I hope this is taken without offence, I feel for you and don't seek to defend the other woman's behaviour.