Retired school teacher here. Just my 2p from a school perspective.
You may not be friends with the parents, but your children may really enjoy each other's company at school. It is possible to make friends with people we like, rather than because our parents like their parents.
Parents who stick together at school and therefore have each other's children regularly, "normally" know each other in another life. "Oh our kids went to nursery together "..." oh we've got older kids together in year 6 (or whatever)". Tbh this doesn't broaden your children's experience. They're sticking within the things they know.. rather than finding independence and confidence in visiting a "different" house, seeing different customs, eating diffetent food etc. Children are very observant...its good to see that some children share rooms/eat at the table/have a garden/grandma lives there too/etc....possibly all the things your child may not have or do. It's just making a broader experience in life for them.
Tbh I don't think there's many parents that enjoy playdates. I know myself even, it wasn't my most favourite part of the week! But my children enjoyed the hospitality of other families and I thought it was important to reciprocate, aside from all the reasons above. My eldest son used to love going to one friends house because the mum used to make loads of crafts with them....something I absolutely hate! I had a playdate that used to love to come to mine because we had a dog and a trampoline and he didn't at home.
Your children are still quite young. You may be pleased as your children move through their school career that they have lots of friends in their circle ,that you've encouraged through playdates.As adults, We all like to have "a mate" that we think is on our side, someone to turn to at work....its no different to children. They all want to have friends and join in and playdates just help those friendships along.
In answer to being worried about your children or your children having the same experiences as you, your children will tell you. My youngest son was terribly homesick and whilst we had playdates at my house, he rarely went to anyone's house. But that was OK and I assured him that it was. I wasn't making him go anywhere. I was terribly homesick as a child myself, I wasn't going to put my own son through it. And I didn't and I was honest about it. I was even honest with the mums who asked him for playdates. But trust your kids, give them the chance to go if they want to.