I'm 50 and like you I've not worked full-time since I was 27. I have had years where I've not worked at all, others where I've done lots of voluntary work and periods (like at the moment) where I work part-time.
Financially I don't need to work, but I'm working part-time:
(a) for the laughs;
(b) to be a role-model for my children - I don't want them to think that sitting at home all day is normal;
(c) to feel connected to my community; and
(d) to feel that I'm giving something back, having benefited from a free university education etc.
I haven't read the whole thread, although I've read lots of it, and given your family history it seems to me entirely reasonable for you to want to hunker down and concentrate on building a good, strong, loving family home for your DCs. I made exactly the same decision in similar circumstances to you.
What I would say is this. Things won't always stay the same. Your DCs will get bigger and need you less. Your friends who are at home looking after their DCs at the moment, will go back to work when they are bigger, leaving you solitary in the day if you are not doing something too. Your relationship with your DH (who sounds lovely!) may not always be what it is at the moment. So it's a good idea to keep your hand in the outside world in some way.
You keep saying you are not good with money and don't understand finance. So how about learning about that? Everyone needs to understand about pensions etc and there are books / on-line courses you could do, to have a general understanding. What about volunteering at your DC's school, even if it's only one morning a fortnight, just to keep in touch? Or just something entirely for you - learn a new language, Art History, anything you fancy really. Sorry if all of this has already been suggested.
You don't need to feel guilty - although I know that it's easy to feel so and to feel that you need to justify yourself. The money you have has given you choices and it sounds as if, at the moment, you and your DH are happy with your choice.