We found it expensive in that there are no cheap alternatives - so prices may be similar to Waitrose, but without an Asda alternative being available, if that makes sense.
Eating out is easily £25 a head for burger and chips, and a beer out is minimum £7.
That said, it's doable. Do a supermarket shop at Bonus (there's one on the main shopping street, and several bigger ones round and about) to stock up on breakfasts and snacks. Avoid 10/11 as this was the most expensive shop.
Hiring a car was cheaper than airport coaches and excursions for two of us, and gave much more flexibility to spend time where we wanted to. We didn't need a 4WD for the Golden Circle in winter, though the weather was relatively mild when we were there. On street parking in Reykjavik was also surprisingly reasonable.
But some beer/wine at the duty free on the way in (and spot all the locals stocking up with trolleys full!)
If you go self catering, you could pack some basic foods in your bags - pasta and sachet sauces, instant porridge etc.
The local pools are brilliant - we chose not to do Blue Lagoon (I had been before, DH wasn't fussed), but the pools more than made up for it. We also found Fontana Laugurvatn reasonably priced (and lovely!) as a spa treat.
There's lots of places on AirBnB, although we ended up with somewhere through Booking.com. Most of central Reykjavik is pretty walkable.