At five if you're using consequences they need to be close to immediate and ideally linked to the misbehaviour.
Research shows that there is no benefit to using larger and larger consequences; something token/small is just as effective as something more serious. It's all about how you use them. And consequences (punishment) are in general less effective than positive reinforcement of the behaviour that you want.
You're right as well that you can't just look at the behaviour on its own, it's important to look at the whole picture, so something like keeping him in all weekend may be counterproductive as he'll be bored/have energy to get out. It's like saying you'd punish a dog by refusing a walk - doesn't work like that. They need exercise and stimulation or these needs will come out in other ways.
In the scenario "Mum thinks dad is too strict/harsh; Dad thinks mum is too soft" (really common scenario) there are two red flag situations to look out for:
Is dad controlling in general? Is there a general sense of contempt for you/your opinions, so that you can't discuss things? Does he escalate to the point where you're actively concerned for the wellbeing of your child? Do you find you're both tiptoeing around his moods and/or you feel like you have to "placate" the child to prevent dad from going off at him? You might be in an abusive situation - abuse isn't always physical.
The other thing to watch out for is when you both respect each other and can have a discussion but you have wildly different ideas about parenting to the point that you each feel genuine concern/distress about the other, and you're each attempting to balance by pulling out - you feel he's too critical and harsh so you try to shower the child in love, praise, acceptance and understanding and empathy, to the point you probably let them get away with too much or have lower expectations from them. Dad feels you're too flexible and unclear and so tries to be extra strong/extra clear about boundaries to try to undo what he sees as a general lack of discipline. May have high expectations of child.
The first is not fixable and you should prioritise safety and consider leaving the relationship. The second IS fixable but you both need to be open to discussing what is probably a highly emotionally charged issue. The key is to see the positives in each other's parenting approach and lean in. So you focus on how to set boundaries and be consistent and raise expectations all in an empathetic/respectful way. And he focuses on how to be more positive or see the child's perspective and maybe some child development. In this way you each lean towards the middle and don't end up with a wildly unbalanced pendulum swing method for the child.