My husband is a Catholic and I am an atheist, over the last 16 years I have ended up spending more time listening to mass than I bargained for (=
It really is up to you to decide if it is something you are comfortable going along to or not, not everybody has the same line in the sand, and some of us find out lines in the sand are more movable than they thought they were. I certainly had no intention of going EVER cos I regarded it as a thing of his that had nothing to do with me ..or us.
But family expectations filtered in and I ended up going to our nephew's first communion, whinging all the way, because allegedly it would look odd to his (entire, huge, gobby) family if I didn't come. I was not a happy bunny going in. But I got to hear the Gloria sung by an amazing choir in combo with the acoustics of Sant'Eustorgio, which has gone down as one of the more memorable moments of my life. Because just for a moment there I got it, I understood why people believe in God. Which is fairly amazing considering that I am as spiritual as your average teabag.
Despite all my firm intentions to the opposite, I had my own son baptized because it was before the rules changed, so at the time unbaptized babies that died didn't go to heaven, and MIL was very very distressed at the idea that our son could end up in a bad place. That was the longest mass ever because I had a very wriggly small person to control who was dressed in so much fluffy stuff that I wasn't sure where his various appendages were. It was like dealing with an enraged octopus having a punch up in a lace factory.
I couldn't have not gone to mass for FIL's funeral, my husband needed me there. And I needed to be there for him. God had nothing to do with it, it was just about love and grief.
I think it is right and proper to decide what works for you and ask for that to be respected, but there may be moments in your shared life where your presence in church next to him might be about so much more than religion, or absence of religion.
Maybe that is what this invite is, a chance for him to make you "offical", and that might be why he is so keen for you to go.