It can be hard. I don't think that necessarily suggests that you're married to the wrong person. It just reflects the fact that life can be hard sometimes.
DH and I have been together for nearly 25 years. During that time, we have moved several times, including across continents, we have become parents, dealt with redundancy and miscarriage, cared for elderly parents, experienced the deaths of parents and siblings, battled a few serious health issues etc. Many of these experiences have put pressure on our relationship at different points, but we have come through them together.
We are very different in many ways. Different culture, social class, religion, even language, so we have had to negotiate and compromise a lot. Sometimes it's really tough. Sometimes he irritates the hell out of me and vice versa. I wouldn't ever characterise our relationship as "easy".
However, "easy" sounds boring as hell to me! The differences between DH and me do make things harder sometimes, but they also help to make life more interesting. I love being able to see the world from a very different perspective. I love the fact that my assumptions are challenged on a regular basis. I love the way in which he frequently pushes me out of my comfort zone. That's what helps me to learn and grow. It helps that our most fundamental values are very similar. Basically, we want the same things. We just have very different ideas on how to get there. That can be challenging but it's also enriching.