Well in your case I think it’s not quite as straightforward as it would usually be.
On the one hand you absolutely do not “have” to let anyone babysit or leave your baby if you don’t want to. End of.
However it’s not a case of you just not wanting to, it’s liked to your anxiety. Which if you don’t address it could get worse and could start to cause you issues.
So no absolutely you don’t have to do anything you’re not happy with but I would consider how you can move forward to be more comfortable with the idea of having times in the future when DC might be looked after by someone else.
If nothing else you may require a medical appointment or something where you couldn’t take a young child. Or you might be poorly or need to visit some else who is poorly in hospital or anything like that.
it would definitely be a good thing to tackle the anxiety whilst you have the time and space to do so, so that in the future you can leave DC because it is likely at some point you will need to.
for what it’s worth, my oldest DC is 5 and I can count on two hands the times my children have been “babysat” - 2 overnights and 4 times for couple of hours each, in 5 years. I can leave them when I really need or want to do something but I just don’t want to!