I'm so sorry to hear your little one is poorly. It's a worry, especially when they are so small.
Formula milk has all the vitamins and minerals a baby needs. Foods like weetabix are very nutrient-poor by comparison. If he is filling up on those, and drinking less milk, at this stage he could become deficient in certain vitamins and minerals, which could affect his development. If he has a bug that is reducing his appetite, this will be even worse.
I don't think anyone here is criticising you to be nasty, they are just concerned that you are getting old-fashioned advice from your health visitor, and this could be causing problems. Early weaning has been implicated for health problems, like food allergies.
If you are reluctant to stop weaning and go back to just milk, could you stop the weetabix, and do things like puréed pear instead? They will give your baby extra fluids, and are quickly digested so he will be more hungry for his milk. Weetabix is a bit hard on the gut when they are so small, and he may be a bit uncomfortable.
Are you giving him cooled boiled water?
I hope he is feeling better very soon.