I don't think there is any controversy that the best health outcomes are seen in babies who have been breastfed exclusively for the first months of life - I don't think there is much difference between 4 mths and 6 mths, except to say there is no nutritional benefit in starting with foods other than breastmilk before 6 mths, and in some studies, there seems to be a slightly higher risk of ill health at 4 mths, impossible to predict in an individual child. There's certainly no evidence that 4 mths is better than 6 mths.
Small amounts of infant formula do change the gut. No controversy about that, either. In susceptible babies, this may have a long-term effect - but we don't really know how great, or who it affects. In a tiny number of highly susceptible babies, it might be an obvious effect. For the vast majority, we'll never know!
Having said that, I honestly don't think that small amounts of formula, given when necessary, should cause distress and anxiety in mothers - they need to understand balance of risks, and be supported to return to full breastfeeding whenever that's possible. If it's not possible, then any breastfeeding is better than no breastfeeding - no controversy there, either.
The public health endorsement of exclusive breastfeeding should be seen not so much as an 'instruction' to mothers, but a standard for HCPs and the rest of society to enable. There is no doubt that many, many babies get formula unnecessarily and that this undermines breastfeeding. This bothers me far more than predominantly breastfed babies getting the odd bit of formula, as mother's choice (as long as the mother is informed).
BTW, Jay Gordon is not the same as struck-off, dishonest, unethical 'Dr' Andrew Wakefield!