Based on what you've said, OP, the hospital feel there's value in waiting for the Covid test to come back because if it's positive they'll be able to evaluate his status then and there, including, if appropriate, keeping him in for any treatment needed. If you leave and it then transpires to be positive, they say he can't be readmitted to them but will have to go to the children's hospital instead. Without knowing what provision is like at your NHS trust, it's impossible for us to say whether there's any risk to your baby in that, although you might be able to weigh it up. But it sounds as though your consultant and his team think the obstetric service might be better than the paeds service in that scenario.
If you think the risk of him being positive is low, the risk of him needing readmission is low, or that the likelihood is that the paeds service will do just as good a job for him, then YWNBU to go, especially, as pp have said, given hosital is probably a more likely place for him to pick it (or anything else) up than at home. You are free to leave, and free to take him with you. However, YABU to say they are lying to you. Hospitals operate on their own time schedule and the lab is probably prioritising test swabs from symptomatic patients. Doctors also have an astonishingly ignorant grasp of how a mother being stuck in hospital can tilt home life on its axis. You may be able to push things along by giving them a deadline for when you will go, but if they're swamped with lab work it probably won't work.
In your shoes, and assuming you haven't left any pertinent information out, I would discharge myself, unless someone can give you a clearer steer on why staying in is better.