I think there's no question that Jamie has changed by the time of Voyager - and that's really the question that Jamie poses to Claire, and so to us: " Sassenach, will ye take me - and risk the man I am, for the sake of the men ye knew?".
Jamie has been changed by what he has gone through and what he has done - and he and Claire suffer the consequences of that. His intentions might have been basically good, or at least not evil, with regard to Geneva and Laoghaire, but the consequences - of not being able to own or care for his son, of his attempt to care for Laoghaire failing - are painful things. It's not like he gets of scot (ha!) free.
I haven't finished Voyager, although I think I know roughly what happens, so it may be that I shift to Team Roger as I go through, but just like Claire, you fall in love, life and bad things and bad decisions happen to you, and you have to deal with the person your lover has become.
So, obviously tv Jamie is lovely to look at, but it may be that good writers will make the character develop, not completely in line with the books, but in a real, suffer the consequences of your decisions, way. That's already happened in season 2, I think.