I had one removed 2 years ago.
I spent 2 years trying to get a GP to believe me, saw a consultant privately, tried a massive steroid injection, which did nothing (in fact I think it made it worse), and finally had it operated on. By the time it was done I could barely walk it was so painful.
It has been a huge relief having it removed, but mine was huge and the pain was incredible.
There are some downsides though. I do sometimes have the same pain as it caused, in the same place, despite the fact that there is no longer a nerve there - it's due to the damaged nerve end that was left behind. I also have tingling and pins and needles. And some parts of my foot (very close to the site to be fair) have no feeling, others are hypersensitive. So there are issues of neuropathy. I don't have no feeling at all in any toe, but I do have a lack of feeling on the side of one, the top of it and part of the side of another.
In terms of fitness - two years later I am still overweight and horrendously unfit. But, I'd been unable to run/jog for over a year before I had the surgery, and found walking (and even standing) extremely painful for about 6 months, so I'd gained lots of weight etc before the surgery. And before I was clear of the 8 week recovery period I was pregnant with DC1. And just when I got back to my pre-preg weight and was looking forward to tackling my neuroma weight and unfitness I got pregnant with DC2 :o I don't think it should be a problem if you're still able to walk etc now and if you don't get pregnant and crave pizza.
I would absolutely have one removed again though. The more you walk on them the larger and more painful they can get and before you know it you're practically immobile. I have had a couple of niggles in the same place on the other foot (common to have them both sides) and I'm going to see a GP asap. It took so long to get it sorted last time that I want to start the process early.