I swear I've replied to a thread almost exactly like this before.
Anyway it's happened to me three or four times. That it's been the wrong time and there was another opportunity later.
First one I got together with the second time and had a problematic but happy relationship until I realised that he was never really going to give me what I wanted, so we broke up again.
Second one there were a few different "almosts" where the timing wasn't right, never worked out, I am glad. He wouldn't have been right for me. However much I mooned over him earlier. He's still a friend.
Third ended up a FWB the second time but I realised I couldn't do it without getting attached, so ended it.
Fourth I married. I think if DH and I had got together when we were younger, it wouldn't have worked. We needed to grow and mature as people first.
I think you're right that falling in love isn't a magical thing, it's about compatibility and timing. The timing isn't right, so leave it. You've done the right thing. If it so happens that everything is right some time in the future between you two, then it might be worth a go, but don't wait around for him - there are plenty more who will be compatible.
I disagree with this however:
When two people meet who can't stop kissing each other or thinking about the other one and have incredible sex and chats that last all night then they obviously have the compatability thing going on a 10 out of 10 scale
What you're describing there is chemistry, not compatibility. You can't tell compatibility from a short interaction like that, it doesn't work that way. Compatibility is about having shared values and belief systems which mesh well together. It takes six months or so of really knowing somebody to gain an idea and about two years to properly work out whether you are compatible.