A lot of what you say about your DSS reminds me of my DS1. I know you say he has no suspected SN, but have you (and your DP) considered looking into techniques designed to help with conditions such as dyspraxia or ASD? Just because he might not actually have the condition doesn't mean that it wouldn't be helpful - my DS has been helped and supported with strategies for ASD for 5 years even though he had no diagnosis, although he was eventually diagnosed with Asperger's last week.
With my DS it seems to be a combination of things - anxiety/fear of getting things wrong (in hindsight this was partly caused by his dad unfortunately), physical difficulties (with fine motor skills especially), not knowing or caring about social convention, not being able to understand that people feel differently to him.
He does the looking over shoulders thing, and sadly I have yet to find a way to tackle it. He seems to have an intense need to know everything, almost as a way of a way of easing anxiety - he genuinely cannot help it or understand why he shouldn't do it or why people don't like him to do it. He doesn't actually want privacy for himself, so he cannot see why anyone else would - my FIL often jokingly says to him "when you get a phone I'm going to read your texts over your shoulder so you know how annoying it is" and DS1 will just shrug and say "that's fine".
TBH I generally avoid reading anything that he shouldn't know about (or that I don't want him to know!) in his presence - I know it doesn't tackle the cause but it avoids the symptom of DS1 learning things he shouldn't. What might help is focusing on personal space rather than specifically reading over shoulders - DS1's school used the concept of space bubbles (ie, everyone has a bubble of personal space around them and you shouldn't step into it) to try and tackle his tendency to get too close to people, but he was slightly younger at the time and I'm not sure how they introduced it - it's possible you could adapt something similar?
Another example is cutlery - DS1 is 8.5 yet his younger siblings' cutlery use is far better than his. It's a combination of him struggling to manipulate the cutlery effectively (so the physical difficulty of it PLUS not wanting to try because he thinks he can't do it), and not actually caring too much about the social convention of cutlery use. So he will usually start the meal using cutlery (but still needs prompting) but will only use either a knife or a fork, never the two together (he pushes food onto his fork with fingers, or holds it with his fingers to cut it with a knife). This will tail off as the meal goes on, because to him it is easier and faster to use his fingers, so what's the point in using cutlery? He doesn't care that it's polite, or what everyone else is doing, or that his hands get filthy, the thing that's important to him is to eat his food as easily and quickly as possible. This is actually a massive improvement - two years ago it was a battle to get him to use it at all.
Most of it does sound like an anxious reaction from your DSS, especially given what you say about his fear of getting things wrong. It might be that when he reacts like this it's better just to bite the bullet and help him, if it is anxiety than repeated requests to do whatever are likely to stress him more and leave him even less likely to do what you're asking of him. With things like the library card, rather than asking him what he will do to keep it safe which puts all the pressure onto him, guidance in the guise of collaborative discussion would probably be better. I definitely think his wanting his Dad to be there for the kids' club is anxiety about what might go wrong - I actually think your DP's suggestion of popping in at lunchtime is a good compromise. This might ease your DSS's anxiety, and DP could tail off his visits if all goes well. The PP's suggestion of a "what's the worst that could happen" chat is good - especially if your DP can make it humorous and get him laughing. How To Talk... has some great strategies for diffusing situations, but it's been a while since I've read it so they might not be suitable in combination with anxiety.
Have you considered a laminated tick chart for his chores? I think something like that might help for personal organisation. I used house points for a while, which worked well for all the kids (must give that another bash actually ) and he could use the points he'd earned for something he covets (my DS1 craves screen time above all else so he usually gets rewarded with DS or Wii time). I know that personal organisation and chores really shouldn't be something that requires reward (I find it really hard to stick to because it goes against everything I've always intended where things like this are concerned - the worst bit is that I have to use it for the younger two as well who would happily muck in without it) but with some kids they just don't do the social learning thing and need an incentive to do it instead.
Sorry for the massive post but hopefully some of it is helpful. Sadly some kids do need micro-managed - my DS doesn't do the "gormless" thing but the opposite. He's a whirlwind without being managed (your example of cereal - DS1 will try to be helpful and make breakfast for everyone while I'm in the shower, but he'll overfill the bowls, get cereal and milk all over the bench and the floor, leave no milk for me to have coffee, and then none of them actually eat even half of what he's put in so it all goes to waste. I've banned him from going into the kitchen when I'm not there tbh ), yet will get distracted by the slightest thing when asked to do something simple like get dressed...