I don't think it is right to tell someone how they should feel to be honest. Or how they should "take" something that affects their lives and their family's
I did not mean this in a patronising or dismissive way. It is easy to get trapped inside a catastrophic mindset (I know because it had totally happened to me too after the referendum), especially with all the EU "grassroots" campaigns that, quite honestly, turned into echo chambers of self pity, full of Nazi comparison tropes as upthread and - sometimes - open hate against the British people. Some of Eastern European "EU rights" social media groups, for example, regularly have extremely racist posts about other, "less deserving", immigrants - Muslims, Africans etc., which get a lot of upvotes. EDL is, paradoxically, capitalising on Brexit and the resulting social alienation, with leaflets now available in Polish - "they want to replace YOU with Mohammed and his twenty wives" (I invented the slogan, the actual one is along the same lines but much more disgusting). The bigotry is on both sides.
I just wanted to offer a different perspective. You are British, your husband and children are British, no one is sewing yellow stars on your clothes. Some stupid people feel entitled to comment because they feel like it's their field day, but there ALWAYS will be stupid people around, in any country, and you won't be able to placate everybody, and neither should you. "The dogs bark, but the caravan moves on" became my motto, and it did improve my life a lot.
I regret the Brexit vote very much, and my forecasts of what is going to happen to the economy is actually more pessimistic than the industry consensus (and I am professionally involved in the crystal ball economic forecasting sector). If I am able to persuade the family court, I will leave the country with my children - I don't see a bright future for them here.
However, to get the facts correct, there was never an absolute freedom of movement between the UK and other EU countries, it was never the equivalent of moving to a neighbouring province. It was always conditional on being economically active, in study or independently wealthy with health insurance in place. The problem was that, unlike many other EU countries, the UK chose not to enforce the rules with registrations and deportations. I find it a bit strange that a lot of people seem to just have relocated to another country, but never bothered to check the rules. But maybe my "visa national" background shows here.