Newmom2be, I can really relate to what you are saying here. I'm now 26 weeks and am so relieved to be finally fitting into maternity clothes.
It's not about the size as such, more the general shape of maternity clothes. For instance, clothes which allow room for a 30+ week bump just look daft when you are 18 - 20 weeks, or at least they did on me! It turns out that I'm not the sort of body shape where you get a nice neat bump all out front, mine sort of spread sideways at first, which made it harder to find clothes to fit.
I got round the problem by wearing non pregnancy trousers, with the button undone and a bump band round my waist and covering the zip area. On my top half I wore vest tops in a size larger than I normally would, topped off with a cardigan which had just one button done up, so my tummy area wasn't constricted. I found a couple of White Stuff tops in John Lewis which accommodated my tummy (again, these were non-maternity, but looser than I would usually wear).
Now I finally have what looks like a 'proper bump' and it's so nice to be wearing maternity clothes. These seem to vary so much in availablility depending on where you live, but I've found Gap good for jeans (they fitted from quite early); H&M good for tops (including some really nice ones which will be good for breastfeeding afterwards) plus some bits in Dorothy Perkins and New Look, neither of which are places I would usually shop, but have some excellent and reasonably priced maternity clothes in my local store.
Good luck, and enjoy your pregnancy. :0)