The thing is, to start with, your children may not be happy. Adult children are like young children, they don't like change, they perceive these things as a threat to the family and would prefer you to be sitting at home waiting around for them to visit!
Over time, though, they may change their perspective and see a happy mum is better than an unhappy one.
I think your partner is correct, you either have a proper relationship, he doesn't want to be your dirty secret any more and he's spent years of his life investing in this.
I would not guess what your adult children would say, and I wouldn't take their first response as the end to the matter. If you are calm, and allow them to be upset/dislike it/wish things were different for a while, they will likely accept it over time and if they didn't, I would not be happy with them either, because sacrificing your life for them when they are mid-twenties adults moved away and not in your daily life is silly.
I think you need to perhaps let one of them in on what's been going on and gently introduce the idea. If they kick off, they will calm down. I've been through this and it does get better over time, especially if you make plenty of time to see them.