I know that fear and it is really hard to start with, but if you approach it carefully, and thoughtfully, it can be incredibly rewarding.
Think hard about the things you used to enjoy pre-marriage and children and start by doing them. Not all at once - it's important to take time just to relax and rest as well. And then get brave and try some new things too. Make a list and refer back to it frequently.
You may find lots of your friends are only too happy to take time away from their families - several of mine have said I'm the perfect excuse for them to have a break! But time alone, and learning to appreciate time alone, is also really important.
I'm child free this weekend - I went for a run yesterday evening then had dinner and read my book. Today I'm going for a really long hike with a friend, then going for cocktails with another friend. Tomorrow I plan to get my nails done then do some baking and run some errands. I may meet a friend for coffee.
I've done all sorts of things I would never have done before - night hiking, going to the opera, a bread making course. Some cost money but lots are cheap or free - the last time I was child free I went to the park with a book and a flask of tea and lay in the shade all day reading. Even things that sound really dull - like sorting out my wardrobe - are fun if they are done through choice and with some upbeat music, and no small children!
I love my child free weekends - really love them - and am a much better and more engaged parent as a result. Guilt free adult time is so important.
Deep breath and make that list! Good luck...