I had exactly the same worries up until about 2 weeks ago and could have written this myself. My baby is almost 4 months old now.
Exactly as you described, my boy smiles and coos a lot too and has done since about 5 and a half weeks (especially when on his changing mat or in his bouncer). He has been hitting all of his other milestones.
He seemed almost nervous making eye contact with me and if he did, it wouldnt be for long. He also rarely looked at me when I was feeding him or cradling him to sleep from a younger age and would seem to squirm to want to get away and cry until I put him down. He even learnt to self soothe to sleep at 2 months old. This made me feel even more concerned, yet my friends and family would say "you should be grateful, it's better than having a needy baby".
He would gaze and smile a lot at my husband or up at the ceiling but would watch me like a hawk when his dad was holding him or he was in his bouncer. He seemed to prefer me from a distance, which upset me a lot.
It wasn't jealousy that I felt, but I wondered what I had or hadn't done for him to be like this with me. I am with him all day and night, I try and interact with him as much as possible when he is awake, I am affectionate towards him, I read and sing to him, take him out for walks, to sensory classes etc. I kept saying to my husband "it's obviously because you're the fun parent" 😄. I tried to pass it off as a joke but it really bothered me.
There was one day that was particularly bad and I broke down to my husband and told him how I was feeling. I even alluded to my fear of autism (not that my feelings would change in any way if that were the case). I have worried about things like this since he was born and because of this I genuinely think I may have been suffering with PND . Although I've tried to enjoy the time with my baby and watch him grow, I've always had some sort of anxieties in the back of my mind- it consumed me and I was thinking about it every day. I even asked my HV about him twirling his foot when he was drinking his bottle and if it was a sign and she said "it's way too early to see any signs of autism, stop worrying!".
What didn't help when a new mum friend I met at a class had implied on several ocassions that my little boy wasn't very alert. She only sees us for an hour a week and I think she forgets that her baby is 2 months older than mine. You shouldn’t compare I know, but I don't think my baby is any less alert or developmentally 'behind' other babies of his age that I have met, but he does tend to daydream sometimes. I think he is a very curious (or nosey like his dad) and is taking everything in.
Anyway, my husband told me (in a nice way) to stop worrying. He said he has no concerns at all and that when he's home he has seen the baby looking and smiling at me a lot and it's obvious he knows where he is safe and comfortable. I really do think I have been overthinking it. My husband works 11 hours a day so I imagine the baby is happy to see a different face that mine. I also read somewhere that babies prefer looking at their dad -something to do with darker, bigger eyebrows, facial hair etc. Who knows?
My husband also said "imagine somebody trying their hardest to get and maintain eye contact with you, it's intense, tiring and can be awkward so the baby isn't going to want a staring contest with you all day". He mentioned that the baby was also probably picking up on that I was trying too hard to get eye contact from him.
Since that conversation I have tried my hardest not let my anxieties take over and have been more relaxed and not trying to 'force' it. I don't know whether it's a coincidence but my little boy coughs or 'shouts' for my attention now, he is the one who is initiating the eye contact, he gives big gummy smiles when I look at him and stares me out when he is being fed or trying to go to sleep. He is currently snoring away on my chest!
Sorry about the long reply, but I hope it reassures you. Talk to someone, it feels like a load off your mind and although easier said than done, put it to the back of your mind and try not to force it!