I can recommend Eilert Smith Hotel in Stavanger - but it’s obviously in the city which might not be the vibe you’re going for. From Stavanger, you are well placed for exploring Lysefjord, plus hiking up Pulpit Rock. Old Stavanger is very picturesque, but I found the rest of the city perfectly pleasant but not particularly ‘wow’.
In Bergen, Bergen Bors is fab but again is in the city.
More ‘rural’ recommendations:
Hotel Ullensvang on Hardangerfjord - right on the fjord, absolutely gorgeous setting.
Utne Hotel on Hardangerfjord - also stunning, bit more basic.
Bolder Sky Lodges, Lysefjord - not good if you’re scared of heights but the most stunning views.
Hauane B&B - not direct on the fjord but further up the mountain - very quaint and homely but welcoming and lovely setting.
If you’d be happy a little further north, along the Sognefjord is a great area to explore.
Fjaerland Fjordstove Hotel, Fjaerland and Kviknes Hotel, Balestrand are both period hotels with amazing views / gorgeous settings
Walaker Hotell, Solvorn is supposed to be the oldest hotel in Norway, again stunning views.