I've always wondered but Google doesn't seem to offer much info. I now have a friend who could be facing the prospect of a sentence and I'm extremely sad for her.
I say a small child because I think it could possibly be extremely upsetting. There's a no touching rule, isn't there? Can you imagine if a mother is in there for a non violent crime and can't even touch their baby/toddler? 
What about breastfeeding? If children are allowed to touch their mum in prison, would they be allowed to continue breastfeeding if they visit a few times a week? I'm sure most would opt to stop feeding but is it possible?