I still think, even despite all this, that it was unwarranted
Really? I don't. Remember he will have told you a very selective account, designed to make you feel sorry for him, and even to give you a bit of a thrill out of "standing by him" and "being loyal".
There's an odd psychological response to doing someone a favour - rather than feeling like the person you have helped "owes" you, you (as the favour-doer) are more likely to feel grateful/indebted to the very person you have just helped*. This weird effect also operates with showing someone loyalty or going out on a limb for them - it makes you feel like you can trust them MORE and makes you eager to demonstrate further sacrifice/trust. Feeling as though you (and you alone) understand him and support him against the injustice of the world can be very seductive - watch out for it in future.
It wasn't easy but I believed in him but and stuck by him
Examine what you just said. It wasn't easy to believe him - because your instincts were screaming at you that something didn't add up. You decided to act as if you believed him, and to squash down your doubts. Why did you do that?
perhaps I should have walked then
Perhaps you should. I'm not saying that to blame you in any way for staying, but to arm you for the future - if you meet a potential partner who has a criminal record but explains to you how it was unfair/not his fault/a technicality/taking the fall to protect someone/someone making up lies about him then consider very strongly walking away at that point and with no further proof.
You don't "owe" a potential (or actual) partner the benefit of the doubt. You aren't compelled to act as if they are 100% trustworthy UNTILL you have solid, undeniable proof that they aren't. It's not somehow "being unfair" not to believe someone if they tell you something that gives you pause. You are allowed to make decisions based on feelings/hunches, on the balance of probability, on the risks involved and for no goddamn reason at all, especially when dating.
Also, well done on changing your number - you are doing AMAZINGLY 
*Incidentally, this is why if you're trying to get someone to like you, you should ask them to do you a favour. Don't offer to do one for them - it's not nearly as effective.