I'm not sure I believe in any afterlife at all, but if I was going to, it would be reincarnation.
Within that belief system, I would say that the 'Observer' is our true self, and the only part of us that survives death.
I would also say that the Observer does indeed change with life events, but in a very slow, indirect way... kind of a process of maturity through gained wisdom and experience.
As we die, and then come back in another life, what makes most sense to me is that all of our specific memories of that single identity are lost, but that we keep the wisdom that we gained from living that life, learning those lessons, and that we move on to new lessons in a new life.
If you think about it, there is actually very little of our lives that we remember even within this life. We think we do, but what we actually remember are major events, or fond memories, or things that were very repetitive (like the way Aunty Jo always gave you a caramel sweet.
Lost in the fog of our brains are the countless days all through toddlerhood, childhood, adolescence and beyond where we just got up and did everyday things. We also forget a huge percentage of our dreams.
Despite all of this forgetting, we retain a sense of self, and we retain a sense of learning and maturing. Even people with certain types of amnesia, who remember nothing about their own identity, seem to still have a certain wisdom that they have gained through their life experience, and almost always remember how to do certain things (that they learned previously).
So, I would say that if we do reincarnate, we would still very much have a sense of being ourselves, and even a wiser and more mature version of ourselves, but with a clean slate to paint a new life upon.