It would depend for me on the sort of house size you're talking about and your general lifestyle.
We used to live in a small terrace which in principle had enough space for us but the lack of living space and space to WFH really ground us down
We now have a separate home office so nice separation of work and home life and a separate kids living room to adult living room. It's great.
I wouldn't sacrifice holidays to have a bigger place but I look back on our previous tiny home and am so grateful for this place.