It's natural for babies to put everything in their mouths, but this is not necessarily a sign of hunger.
They use their lips and tongue (which are very sensitive) to explore objects and gain extra sensory information.
You are probably noticing that your DS does this recently because he is gaining control of his body and is therefore able to find his own mouth better.
For signs of hunger:
Does he cry more often for food?
Does he feed for longer?
Does he seem upset during a feed or at the end, as though he's not get enough to satisfy him?
If the answers to these questions are 'no', then your milk is probably satisfying him.
If he's in the middle of a growth spurt, it is possible that he will want to feed more often, but as long as those extra feeds are satisfying him, there's no need to suppliment your milk with solids or formula.
Your body is designed to increase your milk production in line with the volume he's drinking, so you should find that you can satisfy him, even as he drinks more.
Another factor could be teething.
Although it is unlikely that he will start cutting teeth in the very near future. (They usually start coming through at about 5-6 months) it is possible that the teeth are starting to move inside his gums. This could be causing him pain, which may disturb his sleep or cause him to feed more for the comfort.
If all this has made you think that maybe he doesn't need the solids afterall, I would strongly advise you to stop feeding them to him and to try again when he is closer to 6 months old.
(For more information about solids, you can see an article I wrote here)
If you still think he's hungry, then take it very slowly, and you may want to consider discussing it with your HV.
Good luck.