Absolutely this.
Also, autistic children can sometimes misbehave - for reasons completely unrelated to autism. In the same way that any child can.
So the question is - what's going on with this child's behaviour? Is it emotional dysregulation due to his autism and inability to communicate in that moment, or is he displaying disruptive behaviour due to trauma and/or poor parenting? Or is he just desperate for attention of any kind?
Either way, you don't invite him to the party. It's not fair on the other children, especially those whose birthday it is.
However OP, your mention of "he's fine with me" feels a bit like a stealth boast, or a dismissal of his needs. If he's autistic, then you just haven't been in a 1:1 situation where he's overstimulated. It's much easier to manage an autistic child's needs when you're 1:1 and you can often prevent things getting to the point where they have a meltdown. But you can't magic autism and learning difficulties away, even if you're an excellent caregiver.
My point of this OP is that I think 1:1 time with you would be really helpful for this child, but please be prepared for the fact that he might not always display model behaviour. There may be times when he can't cope or suffers from sensory overload. How will you feel about him if this happens? He needs someone who's not going to give up on him when things go tits up, and if you don't think you'll be able to cope, then it's fine to step away now.
The world is really tough for autistic folk and generally people aren't great at making space for individuals who are different in some way. Growing up is hard at the best of times, but when you're neurodivergent, it can feel like a fucking mountain. And throw in parents who describe their child as "annoying" then the poor kid hasn't got anyone who's helping them learn how to navigate the world and develop appropriate coping skills.
As a general rule, I think it's helpful for parents to teach their children that some others may act differently. But I don't think this extends to allowing their birthday celebrations - or other occasions - to be dominated by a child who has much more challenging needs. Everyone needs the space to relax and enjoy themselves, and that includes your DC.
Your DH has understandably said your house is now out of bounds. Only you and your DH can decide whether you could take your nephew to the park (or somewhere else) with your DC sometimes, or whether you could set up some regular 1:1 time with him. There's no right or wrong answer here - but you do need to be guided by how your own DC feel and make sure that you're not prioritising your nephew over them.
For info - I'm autistic/ADHD and I have two autistic DC, one with quite significant needs.