I can’t go into the reasons here for fear of being pursued any more than I recently have been, but what I would say is tread very, very carefully here and remove the review.
No, it’s not fair, but the loss of income really isn’t just based around those helpful comments, if he can prove (and he can prove it) that you have gone elsewhere to share your review and encourage other people to mark it as helpful, then he has proof of a campaign against him.
And if he has the money, or a friend who is a solicitor, then it’s going to cost him nothing, whereas it will cost you a substantial amount.
I have been legally pursued now by an actual solicitor over one comment which was removed fairly quickly for around six months now. I have had to pay for a solicitor’s advice, I have been physically ill because of it, The individual concerned has shocking reviews online but they have come after me specifically over a comment which was present for maybe an hour.
Trust me when I say that the potential outcome really isn’t worth it.