It's a troubling relationship, yes, but Farrow and Allen never lived together. They maintained two separate households. Andre Previn was Soon-Yi's adoptive father, not Allen.
Having said that, even without the familial complications I'd be very concerned about a relationship between a 22 yo woman and a wealthy, well-known 56yo man.
FWIW, as far as I know they're still together 28 years on, and have been married for 23 of those years. They have two children.