I can understand that....mothers who have had babies who were very ill, or preterm, find it takes a while to stop worrying even when their babies are bouncing with health :)
(And of course it's a parent's job to worry at some level all the time, even when their kids are grown up)
But it seems to me, littlejack, that you would benefit from talking through what happened at the beginning to get it all off your chest, so you are not seeing every developmental stage or behaviour as a reflection on your milk....it sounds exhausting and more than most mothers have to cope with. You can do this by speaking to a good friend, or a health visitor if you have a good one you feel comfortable with, or by calling a breastfeeding line. I'm not talking about an informal chat, really, but a specific chance to talk about everything that happened - you'd need to say to the friend/HV/breastfeeding counsellor 'I think I'd like you to listen to what happened to me, so I can just process it all and move on.'
Up to you - but you sound such a caring, gentle and conscientious mum, who'd find motherhood a lot happier if you could take some of the nagging worries out of breastfeeding :)
PM me if you want some ideas on where to call, or of course you can go to Mumsnet's lists of bf help.