Are you happy going into shops?
Could you plan a shopping list for baking ingredients together, walk to the shop to buy them, and then do the baking when you're home?
Same kind of idea but walking to the shops for some treats and having a cinema afternoon?
Library to borrow books? They're generally fairly quiet still.
Walk to the park with a picnic and a football? You could sit while he kicks a football near you?
Have you got any outside space? Is it enclosed? Could you get some chalks and let him draw on the patio, wall etc? Hopscotch? Small football? Basketball hoop?
Any charity shops nearby that you'd be happy going to for a browse or any toys, books etc?
Don't feel guilty. My ds is 7, nearly 8 now and those toddler years are really hard. They're not quite ready to play on their own and need lots of entertaining and exercising. It does get easier as they get older.
I never did a massive amount of paid for activities with mine, we used to go to the park, library, just for a walk and scoot round the block, getting a bus somewhere, just for the bus journey, walk and cafe, baking, play in the garden