I am rather petite so ordered a uniqlo down coat in Xs and Xxs. I have now tried them both on. The smaller one fits me but feels quite snug when tried on over a thick jumper. What do you think? Should I keep the bigger size?