How do you know about his offence(s) OP?
I have worked with offenders and ex-offenders for years. There are loads of people who have committed sexual offences everywhere. Chances are that you have been served by a sex offender in the supermarket, been driven by one in a taxi and sat next to several on the train. Sexual offences are v common in this country but we don't normally know the person sitting next to us in pizza express has that history, so we dont/cant do anything about it.
I'm passionate about giving ex offenders a chance and helping them back to work etc..
But, I think without knowing the details of the offence, it's hard to say what to do.
I would be happy to shop in a supermarket and use a till where the cashier had a history of sexual offences. But I wouldn't be happy having someone with a sex offence history know the details of my life - how many children I have, their ages, where they go to school, what time we leave for school and return, what after school clubs they do etc... and if you hire this person to work around your home, they will know all this info.
I'm surprised you know this persons history. Employers and others should not be telling you. It would also be very hard for the worker to know that you know their offence, and just as you may not be comfortable with them, they may not be comfortable you knowing - especially as you have kids, as they will likely feel v uncomfortable about it.
There is a reason for confidentiality of these things. Whether this person should be allowed to work with/near kids should be a matter between them and their probation officer. But seeing as you know about it, I personally think it's best (for you, them and your children) to avoid in this case