No - the catholic view (as explained to me by my priest when I was preparing for marriage) is that sex is a very important part of a loving marriage. Thus, if you're not using contraception (regardless of whether you are able to conceive or not) you are open to the possibility of creating life, even if that possibility is only a faint chance. So, sex, even where there is no/little chance of conceiving due to fertility problems is not trivial as it is an important part of a marriage.
Think of St Elizabeth conceiving after many years of being thought barren.