As you say they are only like this together, is either one of them also like this with other children?
Because if it's only happening together, then it might be that, like beerthief says, they are learning about how far they can take physicality and this is being addressed between other children, so they are learning what is generally socially acceptable, but at home they have this sense that "it's my brother so it's fine".
In this case it is probably a good idea to have clearer lines about what is and isn't acceptable, rather than ignoring (which might mean sanctions for certain behaviour or it might be specifically targeting a replacement behaviour to encourage or both).
If it happens more generally, it might be that one is setting the other one off and this is escalating frequently. This is a dynamic I notice between my younger DC sometimes - basically one (usually the 5yo) gets an idea in his head that the other (2.5) is going to do something he doesn't like (for example, break his lego model).
That might be a rational fear because the 2.5yo is obviously more clumsy than a 5yo in his movements, and also they tend to be attracted to the interesting-looking model the older has built. The problem is that the 5yo will deal with it in an inappropriate way e.g. try to get in first by thumping the 2yo, or try to warn the 2yo off by shouting at him to go away, or block the 2yo completely so he can't access any lego at all.
This obviously then provokes an angry counter-reaction from the 2yo. From his perspective, he's just gone to play with a toy totally innocently and suddenly been shouted at, blocked, or even hit. So he will then react by shouting/hitting back, or doing something to deliberately annoy the 5yo like grab the model he was playing with and throw it, etc.
To the adult looking in, both children are at fault because they are behaving inappropriately. Which is correct. But if you then go to punish them for it what you often get is that the 5yo feels aggrieved because "it wasn't MY fault, I was just trying to get him off my lego!" and the 2yo feels aggrieved because "he hit me first!" They probably WILL figure out alternative methods of conflict management when they get fed up of whatever sanction they end up with for this, assuming no issues in this area etc. But it's a fairly long and headache inducing process to get there, and you might also get breakthrough incidents where one of them is still harbouring resentment towards the other for some previous slight that they retaliated and got told off for, so they just do something anyway almost to even up the score.
(OK, a 2yo probably isn't having that sophisticated of an inner monologue - but a 4yo and 6yo absolutely do).
So one other approach is more laborious in the short term, but gets closer to the root - basically you can go in when you notice any slight hint of conflict, raised voices etc and be a sort of mediator, translating the 5yo's worries to the 2yo and seeing if you can get them to use problem solving skills to find a solution between the two of them. I can't remember what book I read it in but I like the one where you summarise each one's concerns and then say "I wonder if there's a way..." e.g.
"It looks like DS2 wants to keep his model safe, and DS3 also wants to play with Lego. I wonder if there's a way that DS2's model can be safe and DS3 can play with lego too?"
They will normally IME start suggesting solutions when you phrase it like this, even as young as 3 or so. But even if they don't, or you think their solutions aren't very practical, you can then start adding ideas as well. Usually there's a totally simple solution just around them each having their own box/space etc. But the more you do this, you'll start to hear them do it between themselves too, which is really lovely, and also helps them practice conflict resolution skills.
If it does happen to be that one child is typically instigating and it commonly seems to be in pre-emptive defense against something the other one might not have even had a thought of doing, then it might be that it's because the instigator child isn't very good at reading non-verbal communication, and it can actually help a huge amount to take the opportunity as often as you can to pre-emptively sit with them when they play with (any) other child, and start pointing things out to the child who is liable to get anxious about other children's possible behaviour, reassuring them you'll act as a shield to ensure the other child doesn't do the thing, but also sort of coaching them in how to communicate to the other child, and to watch what they are actually doing (rather than jumping to what they might do).