Aw I do sympathise. I started with the guilt the summer before I had my DD. I did as much as possible with DS despite being heavily pregnant, went as far as climbing to the tops of soft play structures with him, all because I was so, I dunno, emotional about the impending change. When his little sis did arrive, bless her, she was really not an easy baby, cried constantly unless on my shoulder, which ended up with me in the depths of despair because I felt so dreadfully torn and guilty all the time.
But listen - DS is now 3 yrs 9 mths and that little screamer that came and disrupted our world is now 19 months, and when I watch the two of them interact and giggle and wrestle and share their own little private jokes, when she comes up to him and purses her lips for a kiss which he sometimes grants, I well up, in a good way, and know that they wouldn't have it any other way.
Hope that helps sweetheart - it's just a period of adjustment for all of you, and you will adjust. Don't forget hormones and sleep deprivation play a part too in how you feel.
PS I too did the thing where I got mum to come over and sit with Eve for a couple of hours so I could take Haydn to a toddler group, just him and me - like the other poster said, he probably didn't notice but it certainly made me feel better!
Hugs
Shereen