On an individual case, yes it's OK, as long as the unvaccinated baby isn't showing symptoms of an illness/known exposure to chicken pox or measles or whatever, and your baby isn't immune-compromised in some way.
There will be lots of individuals who are unvaccinated in the population for various reasons - individual parental beliefs, legitimate medical concern, allergies, missed appointments, international move and the baby was never caught up, age, lots of things. Plenty you would never even know about.
The thing that is important is the herd vaccination rate ie how many people in general in the population are immune. For a lot of diseases this is already high, and so you don't need to worry; your baby is very unlikely to come into contact with someone who is passing on infection markers for that disease, whether they are vaccinated or not.
I haven't read this article so not sure if it is an accurate explanation or not, but the diagram in it has a really good example of why this matters:
https://theconversation.com/what-is-herd-immunity-and-how-many-people-need-to-be-vaccinated-to-protect-a-community-116355
Also remember every vaccine that your baby has it's the first one which offers the most protection, booster doses just top up the protection to be even better/last longer. The only diseases your LO will not have been vaccinated against yet at all, but will be later, are MMR, chicken pox if you opt for that/it gets added to NHS register, flu, and HPV.
Out of those, HPV is irrelevant as it is sexually transmitted, MMR are mostly kept at bay by herd immunity/partial herd immunity, chicken pox and flu are the only things they are likely to come into direct contact with, and it is less common that babies under the age of 6 months do get these - probably mostly because they are not mobile and touching things yet, but we also know that some protection is transferred from mum's own immunity during pregnancy. You might have been offered a flu jab yourself for this reason.