Goat's 'formula' is actually available from Waitrose, but it's not called formula. It's called Goat milk nutrition or some such. Made by nannycare. They can't call it formula because it's not legal to do so as others have said. There are two types - the big tub, which is intended for babies although it doesn't say so as it can't, and the smaller tub which is for 1 year plus.
I used it briefly with DS, who has an intolerance to cow's milk and because we were having huge problems getting him to take any of the prescribable hydrolysed milk or elemental formulas. He seemed a little better on it (less vomiting) but ultimately we had to persist with the hydrolysed formulas as he started to take less and less of it, indicating an intolerance to this as well.
Our Dietician wasn't horrified when I told her about the goat's milk but she did confirm that it was unlikely to address the intolerance as all mammalian milks (apart from human breastmilk) are extremely similar in composition and contain similar proteins (the things that trigger the allergy).
And my, it was expensive - £18 a tin, and that will last you about a week.
Goat formula is available in New Zealand and can be marketed as such (think it's called karicare over there).