I’m in Scotland, buy and make fairisle knitwear and I wear it all year round.
Its not associated with Christmas in most people’s minds. Your friend must just not be familiar with fairisle knitwear.
If you want wool in the style you linked have a look at Toast, Bosie, Harley of Scotland, Barbour, Boden, Eribe, Brora, House of Bruar, The Croft House, Campbell’s of Beauly and Dicks of Edinburgh.
None of them are cheap but pure wool fairisle is not cheap.