I've had a bit of experience working with offenders and ex-offenders in an education capacity, both in and out of the prison setting.
I remember feeling exactly as you do, and once I did actually say to one of the guys I was helping that my worry was exactly that - I sounded 'posh', I had clearly had a decent education and pretty straightforward life, and that I worried it might annoy the offenders. But the one I asked said it wasn't an issue - but what they didn't like were the 'do-gooders' who swanned around like they had all the answers, but actually didn't want to get their hands dirty, and didn't have anything practical to offer them.
@NCforAMA is also right when she says that education privileges in a prison setting are really sought after by most of the prisoners, and I never had any trouble in my (sadly, quite short) time, just a room of enthusiastic learners from all walks of life and of all ages. I vividly remember one young guy who'd always thought he was 'stupid', had dropped out of school, etc. We were working on some basic phonics - literally 3 letter words - but the first time he 'got it', he actually started getting emotional. It's things like that that hit you, when you've had a straightforward life with a decent education - you realise how much you unintentionally have been taking for granted.
It's a sad fact that at least 65% of prisoners are dyslexic, or have some kind of learning difficulty - that, coupled with what is often a 'chaotic' home life, is often the reason why they start truanting from school as young teenagers, and then fall into a life of crime as a means of survival with no qualifications for a 'proper' job, and/or no opportunities to do Btecs or an apprenticeship, and still carrying the barrier of a learning difficulty with them.
The prison environment is daunting - it's noisy all the time, it's smelly with the stale aroma of yesterday's food hanging around, and as an education volunteer, you can't go to the loo without being escorted through the many gates and doors by a PO - you also try and hold on until the scheduled break, because the POs often didn't have the time to come over and let out one volunteer just to go to the loo in the middle of the morning!
So I'll echo what other pp have said - if you do have an interest in helping offenders with some kind of education programme, it's worth considering, and it can be extremely rewarding (as well as frustrating - the system is so stretched, and everything is under-funded and under-staffed).
Well done, @NCforAMA - you are truly an inspiration, and courageous for interacting so honestly on here.