I think it depends on the age of your children, and how close you live to the grandparents. Also whether you bf or ff.
If you breastfeed, then it's very difficult to get any time at all away from your baby until you stop, not least due to sore boobs! For me, this was the first six months. I went out for the odd night out, and even to a sports match, with DH before then, but it felt "wrong" and we returned to a screaming hungry baby (she wouldn't take a bottle), barely worth it. Whereas friends who were forced to bottle-feed from the start were enjoying weekly date nights from very early on.
We managed to get a couple of weekends away before DD was aged 2, but each time I felt a physical pain at being away from her, which it's hard to describe unless you've felt it yourself, but I cried a lot both times. (Bearing in mind that I leave her at nursery quite happily!) I did enjoy myself in the end and was glad I did it.
It also depends on your relationship with the potential babysitters. We've tried agency babysitters a couple of times and didn't have the trust (they called us to come home early - DD is a light sleeper and woke up), so have avoided using them since and relied on grandparents. I trust my own parents much more than PILs but my parents don't really have the confidence to have DD overnight. PILs will happily have DD overnight, but don't always do things the way we like to do them, which leads to worry/stress.
At the moment, DD is not going to bed very early/easily and so we are again going through a phase when it's difficult to go out and leave her. Our neighbour's teenage daughter has offered to babysit, but there's just no way she'd be able to get DD to bed.
All of that said, I would not want to be like one couple I know, who go out too very frequently using grandparents for babysitters, with grandparents driving halfway across the country for the privilege. My feeling is that if you want/expect that kind of help, you should either pay for a babysitter or move to live closer to grandparents. Or, don't have children in the first place!
Now that DD is a bit older, and can speak up for herself, I don't feel so bad when leaving her, so am starting to request more babysitting, and DH and I try to have a day or evening away from her about once a month, which is about right to keep our relationship healthy. More would be better!
Sorry, that's an essay, but my answer is: "it depends"!