You can not consent to sex if you are asleep. And having sex in the past does not mean you consent aitomatically to sex for ever more with that man.