Our situation is slightly different because I am vegetarian and my DS (2 years old) is allergic to dairy, so most of the food in the house is vegan by necessity.
My DS's dietitian (we've seen her 3-4 times in 1.5 years because of my son's varied allergies) did not necessarily seem concerned when I mentioned that we eat vegan at home, but she did seem happier when I said my DS has meat at nursery and with his dad.
In my opinion, there is no benefit to my son eating meat outside of the home. I don't feel it is good for the environment, the animals, or him, but I think that having a balance of meat out of the house/vegan in the home is an acceptable compromise until he can make his own decisions. If he never ate out of the house then his diet would be vegan, but this is too difficult to implement with nursery/grandparents/dad.
For what's it worth, I've not seen a HV in over a year (DS is now 2). So their opinion is really not that important if you have done your research and believe you're doing the best for you DC.
I take most of my information from nutritionfacts.org, it's a fantastic resource and has information about child nutrition.
Oh and while I was breastfeeding I was zero dairy due to DS's allergy, then once I stopped breastfeeding (11 months) my DS went on to Neocate formula (his was prescribed so was free, I've heard a tub can cost £40 a pop), then Alpro soya (don't buy the 'for kids' version, it's full of sugar).
Hope this helps a bit.