Am wearing one of these on recommnedation of the physio but find that the baby doesn't move much when I'm waering it but goes crazy when I take it off. Am concerned that the baby is too restricted. Any ideas / exerience? Thanks