I want to come off the pill as my partner and I rarely have sex. Do I need to ask his permission? He might not want to use a condom if we do have sex eventually.