But he DID hit you, that's the thing! And I think that pretty much everyone can tell the difference between an angry slap and a playful swat, especially the 'giver'. Not that even a 'playful swat' is OK if the recipient doesn't like it.
So I guess I'll have to add it to the list of things 'that are not abuse because I didn't hit you' according to my ex. Let's see, that's pushing, shoving, grabbing, 'giving a shake', tripping, and now a hard slap on the bum. Oh yeah, and slapping 'isn't that bad, it's not like it was a closed fist'.
Even if it had been a 'playful slap' where he misjudged the distance and/or velocity he should have been full of heartfelt apologies for having hurt you. And a promise that he would NEVER do it again. Instead he's turning it around on you. And why would you know that he would 'never do that'? It's up to him to prove to you that he would never do that by, well, never doing that.
Your decision of course but I'd be taking a long, hard look at the relationship.