Dunno, but my left boob is noticeably bigger than the right after only 2 weeks. Bizarrely though, I find it easier to express (in terms of quantity) from the right hand one?!
Can't help with the "should you be worried" bit, but at least you're not alone