Ah this is useful to know. I was picking up bits of what you were saying about struggling with baby groups and not wanting him to struggle with social skills and I wondered if there was something there, but didn't want to suggest if it wasn't already on your radar, as it sounds like you have a lot to think about already.
FWIW, IME (where the family history is more ADHD leaning) schools/nurseries have actually been much better able to support where I've been upfront about this. Although I was a bit alarmed when I first switched to this approach and they were asking me questions like whether my (then 6yo) could speak, and worried I had done the wrong thing, it was actually so much the right thing. There is a massive difference for DS where he is perceived as having difficulty and supported vs where he has been perceived as troublemaking, wilful or aloof. He went to 3 different settings before he started school at 7 (not in UK) and I did this exact thing with the second one, even though we had moved him from the first (large, busy, long commute) setting due to overwhelm, I chose not to mention concerns in order to give him a fresh start. The only thing I told them was that he could be "sensitive". The staff at this place were absolutely lovely, very experienced and so kind and I don't think they did anything wrong, but it was a bit of a disaster because they made assumptions about what he would be able to manage due to his age, and then misinterpreted his cues of stress/overwhelm as him not wanting to engage with them or just being naughty/disruptive. And because they thought he was just having settling in issues they didn't mention it to me straight away. I also didn't know as much then, so I didn't pick up on cues I would probably notice now in hindsight. I think really I was worried they would think if I mentioned ADHD (which is what we thought at that point, he is now diagnosed ADHD, with a question mark over AuDHD) that I was looking for a label to excuse his behaviour rather than dealing with it, which was absolutely not the case.
The other thing I have learnt for DS is that he doesn't always pick up on things in the way other children would. He didn't especially have issues with violent behaviour, at least not at nursery/school, though he could be boisterous/disruptive, but telling him "kind hands" would not have made any sense to him or helped him understand what was expected instead. He needed really clear guidance both to show him when behaviour was unacceptable but also how to approach other children to ask them to play, for example. And I'm now wondering whether the random pushing while looking happy is possibly your DS' way of trying to initiate a game - something which has been helpful for my DS has been teaching him very formulaic games that other children will understand, such as catch/tig, hide and seek, or a particular favourite activity at nursery e.g. doll's house, ice cream stand, train track, and a simple phrase for asking for it. When he was feeling self conscious or anxious it was like he would lose a lot of his verbal capacity so for example at pick up, (at age 4) rather than say "DS3 has an apple, I want an apple as well, can I have one?" he would just make a whining noise and point at the apple and if pushed just say "Apple". If I had known then what I know now, I probably would have looked into whether he could use something like a set of PECS symbols on a keyring to support him in those moments, but this kind of thing wasn't on our radar at that point. I was trying to encourage him to speak to staff members and use full sentences, which he could do when he was feeling more together, but at that point of the day I do think it was genuinely beyond him at that age. And that is not typical, but he was very very overwhelmed and distressed by that setting, again despite kind, caring and knowledgable staff who really wanted to support him.
I wonder if at home you can try something like thinking up a gesture and short phrase, like putting a hand on his shoulder and asking "Want to play?" in order to initiate the more physical games that he likes, prime all the family members so they start doing it too, and then let nursery know that you've started this at home in order to show him an acceptable way of asking someone to play, in case it's helpful (or in case he tries it and they didn't understand the intention).
There is nothing wrong with being ND but it does help if people understand that it's possible he has a different approach or a different timescale rather than trying to coax him into a neurotypical one or waiting for him to pick things up by inference. IME, there is quite a lot of wanting to reassure you that your child is not autistic or "it's too early to say" (esp for ADHD) if you raise the possibility, rather than being mentally put into a box, but a lack of sharing this info mostly leads people to assume behaviour rather than difficulty, which I didn't appreciate is because a lot of the time children displaying these behaviours have had more lax rules at home and do turn things around when they are shown clear boundaries and settle into the routine of nursery. I guess I didn't recognise how unusual it is when a child does not respond to those methods. I just thought they have seen so many children so they will know what to do, but the majority of children respond very differently to DS, so staff can be a bit thrown if their usual approach isn't helping, and they often don't have enough experience of ND children to be able to recognise profiles, especially as many children are diagnosed later than nursery age, so they may only be aware of how children could present in a very narrow/stereotypical way. It's also arguably correct for them to avoid suggesting concerns in this direction, because they are not trained medical professionals. OTOH, we only know our own child(ren) and what is normal for them, so it can be difficult to know whether they are typical among children their age or not. It took me quite a while to recognise that what is normal for DS is different to other children his age. Even though I have 3 children, the older 2 both have ADHD (DS1 would not have been diagnosed at this age) and I concede I have no sense of where DS3 is in relation to the "norm" any more.