They don't notice or care who is in the 'correct' seat within groups, especially if the plane is anywhere near full. DP and I never look to see which is my seat and which is his seat, so the odds are that we get it wrong half the time
You should sit in your allocated seat for take-off and landing so that in the event of a crash (those are the riskiest times) the airline know who was in which seat. Useful for passenger tally
I think that they will use something a little more conclusive like DNA, dental records and a general description of our charred remains to differentiate between me and DP should the worst happen, rather than where we were sitting before the plane got broken up into millions of pieces.
It's only small or underoccupied planes where they care where you sit for balance/safety reasons. For full, large jumbo jets, there's enough averaging and randomness to ensure that the plane is balanced, although I'm sure the crew have a general look to make sure that one side isn't full of people who are visibly much larger than those on the other side.
For very small planes, they need to know whether you are male or female because on average, a group of men will weigh more than a group of women and I have seen people being asked to move on small planes with lots of spare seats if the plane didn't look balanced.