Parenting is hard - when mine were that age, I absolutely preferred being at work!
In terms of whether you are a good mom or not, ask yourself these questions (not expecting you to answer on here):
Does your child have clean clothes that fit them?
Does your child receive food and drinks that are nutritious?
Does your child have a bed to sleep in?
Does your child have a safe home?
Does your child have age appropriate toys?
Do you take your child to activities outside of the house?
If your child comes to you for a hug - do you hug them back?
Do you give your child spontaneous hugs?
Do you talk to them and interact with them?
Do you comfort them when they are upset and hurt?
I'm assuming the answer to the first set of questions is yes, from your answers on this thread. If the answer to the second lot of questions is also yes, then you are a good mom and you just need to figure out how to address the fear that your aren't because it isn't rooted in reality.
If the answer to any of those questions is no, then you need to seek some support.
Your HV is correct, that social services will only get involved if there is a safeguarding issue, but you can ask for support from Early Help services for which safeguarding is not a criteria. Search Early Help or Family Hubs in your area if you think you do need extra support - you can self refer (HVs are notoriously shite at referring to Early Help services)