Yanbu to feel how you feel, ime we can't help that.
I'm a single parent, my mum helps with childcare (for free) when I'm at work, that means she picks up/drops off/does overnights once or twice per week as I work shifts, none of which fit in with school hours.
My ex also has the dc most Sundays, and (supposedly) every other Saturday.
All of that, may look to someone on the outside that I am very lucky, I have lots of 'time to myself' (mostly spent doing housework/cooking/helping friends with childcare.
But I'm jealous too. I'm jealous of people who have secure loving relationships, I'm jealous of people who spend their weekends as a 'family' eg mum and dad. I'm jealous of people who don't have the other parent always bad mouthjng them, and threatening to go for full custody (or whatever the term is now)
Instead of dwelling on this, I try and look at what I HAVE got, rather than what I haven't :)