Just go there. Make it early in the morning, as that makes it less likely anybody you know will have got up by then. Eight o'clock is a good time, as people getting ready to finish and leave for work are going, but a lot who would come later haven't got there yet.
Take your phone, keys, a bottle of water and a small towel (for wiping your face/hands, there will be sprays and paper towels for cleaning the machines before and after you use them - nobody likes sitting on somebody else's bum sweat). Later, if you're comfortable with it, you can bring stuff for a shower/a change of clothes, depending upon what their Covid safety rules are, as some don't want people changing there.They will also usually sell padlocks for the lockers.
But you need to join the gym first. To do this;
Go to the front desk and tell them you're interested in joining. They'll probably get one of the staff/trainers to come and show you around. They're really nice and friendly.
If you like what you see - it's clean, not too loud and you can see yourself being comfortable there - you can ask about cost, class timetables and booking an induction session, which is when one of the staff spend about half an hour with you, talking about a good start plan and show you how to use some of the equipment. Any time when you're there and aren't sure on something, as long as a staff member isn't actually with somebody else, they'll be happy to help you; it's what they're there for.
Before long, you'll be so comfortable, you'll wonder why you were so scared in the first place.
In terms of clothing, you'll need comfortable leggings/shorts, a good sports bra, a top, stretchy knickers that cover your bum are more comfortable than little cottony ones, socks and trainers, preferably ones you don't wear outside the gym so they're clean. You'll need to tie your hair back and it is far more comfortable to have a change of clothes for afterwards than to leave with your exercise gear on if they allow showers and changing - especially underwear and socks. I'd also suggest you get a pair of sliders for the shower or standing on the changing room floor.
If they allow showers/swimming, bring shower gel and a bigger towel in your bag. It's a good idea to have a second waterproof bag inside to keep wet/damp/sweaty clothes separate. You'll also need a comb from detangling your hair.
Anything new can be scary. But they will be really nice to you and it's a safe place where they won't tolerate anybody being mean to you.