If you genuinely think the teacher hit the child, you should contact the police.
Do bear in mind, though, that an accidental blow from behind could be difficult for a child to interpret correctly, particularly if it gave them a bit of a fright. The swearing is bad if it's accurately reported - he shouldn't be using ''bloody', 'idiot', 'for God's sake', and this alone is reason to withdraw them although not reason for a police report. 'Dopey' I think is a grey area depending on context and tone.
Ignore the nonsense about contacting his regulatory body! There are organisations he might belong to, but none of them fulfil the function of a regulatory body and none of them have any powers to take complaints/investigate individuals - effectively, they are mostly trade unions. Your route for redress if an assault is committed is via the police. This has the added benefit that it will show up on his DBS check in future.
If he's part of a piano school you can of course complain to the school.
Two other things stand out from your post. One is that, if your children are genuinely significantly uncomfortable, or if you are getting bad feelings about a teacher, you need to be a lot tougher about how you step in to protect them - you shouldn't be needing to ask 'how do I go about cancelling' because you are the adult, and in a worrying situation, it's your job to step in to protect your children, and I say this as a music teacher.
The second is the information about the debt. Don't mention this if you refuse to pay your notice period, because it isn't relevant and does muddy the waters. If a teacher is swearing at and even hitting pupils, you have good reason to withdraw them immediately and breach the contract in regards the notice period. He is unlikely to challenge it if his conduct has been questionable, and if he does challenge it, you can quite reasonably say you had no choice but to withdraw the children immediately and you can't be held to the notice clause under the circumstances.
However, if the situation has been exaggerated, you would probably still be liable for the notice period and the fact that you're in too much debt to be able to really afford these lessons is neither here nor there. If you mention the debt, you risk people suspecting you're inventing allegations to get out of paying for lessons you can't afford and shouldn't have signed up for in the first place. If your children are actually being mistreated, you don't want to muddy the waters this way.
Whether an agreement was signed or not is irrelevant. If he sent you the T&Cs - which he must have done, as you're aware of the notice period clause - and you continued with the lessons, you've accepted them by default. The time to challenge them was when you got informed of them. Your only grounds for defaulting on notice will be if he is mistreating your children.