OP, I'm sorry for your loss, but if you're not keen, there should be an end of it, for me. I'm really not in favour of naming children after the living or the dead for the sake of it. It would be a nice tribute to your brother's memory if you liked the name enough to use it, but you can't fence off a child's actual name with air quotes saying 'I don't actually like this, but it's to honour a relative's memory'. The child would just be Matthew Russell Jones, not Matthew 'Russell-in-memory-of-my-brother' Jones.
Having said that, it works rather nicely with Matthew as a middle name. Do you really hate it?