We sound similar in colouring, not sure why you find it difficult, when we get it just right it really looks nice!
I suit Navy blue, very pale or charcoal grey (not mid), pink (this is not a bad thing), lilac, white, breton stripes, small neat patterns, yellow, black, burgundy, mixed reads (but not a flat red for some reason).
Most of my stuff is navy, grey and white, but then i like the simplicity of that and don't feel restricted.
I look terrible in orange, brown, mid grey, cream, tweed and green.
Also try to figure out if you are cool or warm toned (skin/hair). I am cool toned, so if i wear a red, whether lipstick or tshirt it is best to match a cool red than a warm, more orangey red.I hope that makes sense! Also applies if you colour your hair, don't go for warm tones if you are a cool.
I am pale, but just a touch, and I mean barely a touch of dove summer glow (for fair skin) moisturised into my legs after a shower gives a nice bit of colour to take that deathly grey-white paleness away. It is pretty fuss free and I only apply it once after a shower and it is dry in a few hours. It doesn't go orange or streaky for me and i am so pale you can almost see through me.