I have ADHD. It can be tough on relationships, but the flip side is, don't get sucked into everything being related back to the ADHD - sometimes it's just a person's personality.
IME forgetfulness is absolutely normal and par for the course. If he isn't willing to take medication for it, don't expect that to get better. You will have to work around it. Anxiety can be a comorbid diagnosis. Some ADHD patients find that anti anxiety medication is helpful to them. Some have developed the anxiety as a result of making constant mistakes and receiving criticism, etc.
My biggest piece of advice would be to try and forget "disorder" and instead think "person" but just be aware that in some areas he's absolutely going to be on the edge of the bell curve, and he might not be able to help that. So whereas people might say "Oh if he forgets your birthday, he clearly doesn't care!" that's not necessarily true; his memory probably just is that poor, it's not a useful gauge of how much he cares in this instance. But, he should show that he cares in other ways. You might have to look in unconventional places to find it. But that can end up being something you love about the person.
But yes, when you're thinking "person" rather than "disorder", that also means that you're allowed to say, you know what, actually this insecurity/messiness/forgetfulness just doesn't work for me, I really need a partner who is more reliable and organised. You must not feel guilty about that. Only stay with him if you can genuinely handle it. It really puts an enormous strain on a relationship if you're just pretending to be okay with these things because you think it's unfair not to be okay with them.
He should take responsibility for his own issues in the sense of insecurity and anxiety. It's okay to lean on a partner for support, but it's not okay to expect a lot of a partner in terms of alleviating anxiety and insecurity. If you find you're curtailing your activities or acting differently in order to placate his insecurity, that's a problem, and he should really be looking for support with it outside of the relationship. Certainly, if that kind of thing continues and/or he's unwilling to seek help in that area it's not a good idea to continue the relationship.
People with ADHD are impulsive as you have found and can feel things quite intensely. You might need to be the "brakes" in the relationship. Fast moving relationships are a red flag not only when the person trying to move the relationship has nefarious intentions. It's also risky and unhealthy to force a relationship into deeper territory when you haven't mastered the shallower depths first. Especially as you have a child involved, you may need to put up boundaries here. Seeing each other constantly is a bit of a red flag - again not necessarily that his intentions are bad - but that he might be obsessing a little bit over you which isn't healthy for the relationship. It's important for you both to maintain outside interests and take things slowly if this is going to last.
Don't go all out into research into ADHD in relationships as this is such early days, it's inappropriate. Give it six months or so to get to know his quirks, his patterns, which bell curve ends he's on and see how you feel and where you think it's headed. Then if things are still positive you could look into some more in depth information, but I think it's important to keep a sense of distance to begin with, in any relationship, but with ADHD involved you might have to add this sort of superficially, if that makes sense.
Hope that helps and feel free to ask anything.
thanks.