yes it is ok for them to be living in 'shanty' housing - it's usually a lot better than what the unemployed are living in. Usually these 'shanty' houses have electricity, access to clean water, and a little 'plot' on which to grown their own staple foods (often maize in Africa). These days many of the bigger commercail farmers have schools built for the farm workers children and really those houses are no worse than what a large % of the rest of the population would be living in.
In fact of the company owned houses on farms that I've seen most are BETTER than the high density suburbs - and it certainly beats the traditional village huts that most younger people are depeserate to get out of these days.
I think we forget that while proper Fair Trade is FANTASTIC. Even 'normal' working conditions for people in a countries where unemployment levels are higher than we in the West could ever imagine are better than no work at all.........