I do sympathise with your problem, the first time I left DD with a babysitter I was a wreck. She was not bothered at all, had a great time with her auntie.
Tbh, I do believe it is important that you and your dh go our together occasionally, on your own. But, it is probably not a good idea to be so rushed into it. I would also be wary of a babysitter that my DD had never met previously.
How would you feel if you were to organise a babysitter of your own chosing, to babysit at home in familiar surroundings to your babysitter and go out for a few hours?
We always ask the babysitter to come an hour before bedtime, she plays with the DCs and puts them to bed. It means paying an extra hour, but if they do waken up, then they know that the babysitter is there, and not Mama.
In this instance, I would agree with the other posters, go with your own instincts, maybe go out for a drink with your friends then go back.
Or you could phone before you go the restaurant and ask if everything is ok. If the restaurant is not to far away, the babysitter could ring you if there was a problem. Would that work?