I think you look great in both! The dark sweater comination looks neater, but the beige as if it feels snugglier.
So, superficially, yes, the beige is too big/making you look bigger; however psychological comfort is also important (possibly more so, for me); plus I think you have to factor in colour too: if you were wearing oversized charcoal, and fitted beige, the effect might reverse - the lighter colour is more prominent irrespective of sizing.
(Sorry for appalling sentence construction, it is a day of very little brain, but I so relate to horror of sudden squishy front and middle x )