OP said I get really embarrassed whipping out the colour swatches. I worry I look like a middle aged cliche or a shoplifter
...and that made me laugh. Yes, it can be a bit, um, unusual to whip out a colour wallet. However, having colours done really can make a difference so I don't see the difference between whipping out a colour wallet vs reading labels in the grocery store to determine allergens or sugar content or whatever it is one needs to know. You're shopping intelligently and using whatever "tools" you have to make the right purchase.
Plus, it's not just the middle-aged who are interested in fashion, in case you hadn't noticed.
After I had my colours done I gave my husband the same for his birthday. He absolutely LOVED it, ultimately, and has been known to whip out his colour wallet on Savile Row (albeit sometimes to the bemusement of the shopkeeper).
I have a friend who is so enamoured of HoC that she had her children's colours done, and it's actually very sweet seeing them at the shops whipping out their own colour wallets. (Oh Gawd, again, this all sounds so elitist and hoity-toity.)
The point is, colours are part of nature and we are part of nature and why not make ourselves as visually interesting and confident as we can? Dressing in colours doesn't have to be a CBeebies Festival--it can be daring, sexy, bold, powerful, whimsical, fun, even stern (ever seen a woman in head-to-toe red taking no prisoners? I have; it's glorious).
As for shoes, here's something I'd like to whisper...for a Spring, you're going to be in the brown/tan/navy camp rather than the classic black, but...sometimes I actually purchase and wear clothes that are outside my palette. For me, having colours done is a great way to:
a) know which colours look amazing on you
b) know which colours definitely not to wear
c) give you more information to make decisions for your own look, but with more knowledge about what gives you your best.
HTH