I have done a lot of reading.
In theory:-
Punishment isn't very effective on its own anyway. We tend to think it is effective because it feels rewarding (the sense of "justice" is apparently very reinforcing to the person who has been wronged and those who sympathise with them) but in actual fact, what changes behaviour over time is development of the positive opposite replacement behaviour, aka skills, whether that's emotional regulation, social communication, academic skills, whatever it is. 99% of the time when we feel that punishment has "worked" to change a behaviour we are seeing the effects of the skill-building or replacement behaviour side.
Some skill development will happen naturally over time as a child matures and their brain develops and they gain more experience/practice of the situation and try different ways of handling it and perhaps get rewarded/encouraged for the socially accepted behaviour. For example, most children learn to walk as they develop the strength and coordination to do so, without being explicitly taught because walking in itself is inherently rewarding in that it affords them more freedom/mobility.
Some children need more targeted support due to SEN or other difficulties and/or learned survival/coping behaviours which are serving them badly. In order to target support, you need to identify the positive opposite replacement behaviour, which might involve working out what they are getting from the current unwanted behaviour, break it down into steps from where you are now to where you want to be if necessary, and then work on scaffolding (supporting) them in having successful experiences demonstrating that behaviour/skill, which might include changing the environment, better prompting (e.g. avoiding "Stop/Don't" instructions, reminding of specific expectations rather than a vague "be good") and/or a lot of reinforcement (rewards, praise, encouragement) to mark any movement towards the preferred behaviour. This is the most effective way to change a behaviour or develop a skill. Punishment doesn't really contribute much to this, just as we know that punishment doesn't help somebody learn to read or do maths calculations better. The effectiveness of punishment doesn't scale with the severity, so if you must use it, something mild is the best balance between effectiveness and downsides.
If a behaviour intervention isn't working to help change the behaviour then the most effective approach is not to adjust the reward/punishment but to look at the gap between the current behaviour and the expected/positive opposite behaviour and reduce the gap by adding a stepping stone or if there have already been steps broken down, look at making the steps smaller. This may involve looking more closely at the typical process of development of the skill you are trying to improve, as we do in learning support - for example, a child who struggles with handwriting may need more fine motor skills practice. But this kind of approach can work in a lot of behaviour type situations as well. For example, a child who hits we might have the expectation that they use language to resolve a conflict instead. But before somebody can use language to resolve a conflict, they need two underlying skillsets: They must be able to speak clearly so that others can understand them (so speech therapy may help) AND they need to be able to understand and predict social behaviour.
Also, it's important to understand that stress, including physical stressors such as tiredness, hunger, discomfort, and what they call a "hot" emotional state (ie, agitated due to any kind of emotion) affects the executive functions which is the part of the brain in humans which allows us to self-monitor and direct behaviour towards a future outcome or wider goal. (This is in development throughout the whole of childhood through adolescence). So if a child is sometimes able to do something the "right way" but when they are anxious, excited, or need the toilet it seems like those abilities go out of the window you might be seeing this in effect.
In that case it can help to either take that into account when you are looking at behaviour expectations - we do this naturally e.g. most people would expect a bit less of DC or let more things go when they are tired.
Or if it's a chronic problem which seems to be leading to frequent or intense behaviours then you might incorporate this into the "look at the environment" step. Can any of the stressors be reduced or removed? Are there cognitive or other developmental aspects they are struggling with - e.g. can they actually keep a multi-step instruction in mind or could this be given to them step by step? If this child has sensory needs, are they being addressed? Sometimes a "sensory diet" can help a child come from a "hot" state into a "cooler" executive function state. If this child is struggling with emotional self-regulation, do they have access to a self-regulated, trusted adult to co-regulate with? If they have ADHD and therefore their executive functions are not communicating well with the rest of their brain would medication be appropriate? Do they need glasses, is their hearing OK? Are there sources of pain or discomfort such as dietary intolerance, vitamin deficiency or muscle weakness?
In practice:- (My DS nearly 7 is similar).
Aaaaaaaaaaaargh WTF, I don't know, it's so so hard. I have all this knowledge but it feels impossible to follow all of it day to day. Progress is slow. You feel like you get somewhere with one scenario/issue but the others haven't even moved the needle or a new one appears out of seemingly nowhere. It can take several days to find a calm enough point to try to speak to yout child about a behaviour by which point it feels pointless because they have probably forgotten about it totally. No idea how to navigate the fact he doesn't seem to understand the typical social concepts like someone being cross or upset and frequently sees no issue with the way he has behaved. Professional help is baffling because they tell you things like "Yes your child has ADHD. Make sure they have structure at home!" errrr thanks... I thought we already did but no, let's read loads more and talk to other parents and try to figure out what exactly is meant by "structure" in this context and what might be helpful. Some things help, some seem to make no difference at all and some seem to increase stress and therefore stress behaviours. I do have a light at the end of the tunnel, in that my older son was similar, I had no diagnosis or context and barely any of this knowledge, did 90% of everything wrong, the 10% I did get right (I think) was trying to be compassionate and understanding, and he did grow up and out of it anyway.
Occupational Therapy seems to be helping, if you have access to that at all. Some things I have tried to teach/explain to him in the context of skill building he has not been receptive to until his OT brought it up. ChatGPT is a surprising win in terms of - when I want to know what skill is underlying a particular behaviour expectation I have, I have asked here and it usually seems to give me the right term, which I would never have found by googling, but when searching in reverse and cross referencing, has always been correct so far. Then it can suggest activities to practice and develop that skill, which is extremely helpful, though again slow progress. And if it's a load of rubbish, then at least it's positive fun new ideas for playtime together.
The book Big Baffling Behaviours helped. The Ross Greene Collaborative Problem Solving framework kind of helps - tricky to do directly with child, but I have still taken a lot of useful ideas from it. Mona Delahooke helped. Declarative Language can help, when I'm not in a rage about it being such an annoying concept 
Getting my own ADHD recognised and treated and as a result having a much calmer and more functional household helped.
Breaks for the parent absorbing the majority of the dysregulation are an ABSOLUTE NECESSITY. A group of parents (online or IRL) going through anything even close who you can swap memes and exhausted jokes with helps. A few people who see the positives in your child are invaluable because a lot won't see it.
And apparently the compassion and trying is important even if you get everything else wrong. So hang in there 