IMO context is everything.
Bastard became a 'bad' word because in days of yore it was considered 'bad' to be born outside of wedlock and very poor form for the mother (tsk, tsk, naughty harlotty unchristian woman...).
So it is now a 'bad' word, or a 'swear' word, that can be used to insult someone.
BUT being a 'bastard' isn't such an unacceptable thing to be these days, especially among the younger generations. My DD, from the 'wrong side of the sheets' as some people describe it, or 'my unexpected blessing' as I like to describe her, doesn't have a problem being illegitimate.
She does have a massive problem that a word, which is considered to be so offensive (eg a 'swear word') could be used to describe her because of the circumstances of her birth. It's not the actual word she finds hard, but that the word has now become a bad word, eg people tend to use it to describe an unpleasant person, or someone who has been completely unreasonable (utter bastard). She doesn't mind being considered illegitimate, it is a fact. She HATES thinking that she could be thought of as an unpleasant person (sort of how the word is used today).
Context is everything - I don't think it's acceptable to use as a term of reference because it has insulting overtones, rather like me not expecting people to refer to my lady bits as a cunt lady bit, even though it is technically that, as that term has insulting connotations.