We ended up in children's A&E with both of ours. DD is lactose intolerant and multiple GP visits resulted in the GP treating the symptoms, not the cause. The day she finally refused to take any formula all day was the day we took her in. The paediatrician we saw was lovely, listened to DH's lactose-intolerant family history and prescribed lactose-free formula. DD was like a brand new baby within days. Feeding had been a real battle up to that point (4.5 months' old) but became easy and she was clearly much, much happier.
With DS, we saw the signs of intolerance and tried him on DD's LF formula (13 month gap, so we stil had some left over) which didn't help him, so we went to the GP who referred us straight to the paediatric team and told us that if he started to refuse feeds before our appointment, to access the team via children's A&E. Which is what happened.
I think where there's known intolerance / allergy in previous children they're more likely to take you seriously with subsequent children, rather than being written off as neurotic first time parents (which is what our GP did).
As long as your baby is still feeding I probably wouldn't go to A&E. But, if nobody will refer you to the paediatric team then A&E may be your only way into them, which is daft given the history.
In our area the HVs can refer, and ours was going to refer DS if the GP didn't. Could you see if they're able to do that this week for you? We had baby weigh clinics every day around the town, so HVs were easy to access. Hopefully yours are too?