You are NOT being unreasonable to say you're not happy but it does indicate TRUST ISSUES on your side of the fence.
You have every right to tell him about your concerns but he has every right to blow back at you for your lack of sympathy about his working week.
I spent years doing much the same thing. DP had insisted on moving out of London for a rural lifestyle. Turned out there wasn't much of my type of employment anywhere near.
Sometimes I could get away with a 2+ hour commute each way, each day but that's such hard work after your thirties but it was not always possible.
I tried coming home one night a week, but that means packing and lugging your cases to/from a hotel near daily. I ended up becoming friendly with and confidante to the bar-staff just as a way of having social interaction. Tbh, the nightly phone calls were more depressing than uplifting hearing all the enjoyments others had been having. It wasn't helped by being expected tonuse my weekends to catch up on all the stuff around the house that needed doing.
But, although I stayed at.most remote jobs for 2-4 years it was always in 3/6 month increments, never 100% predictable. So quite tricky to sign rentals or buy a place to call home
It is intensely frustrating to live a lifestyle where you live out of a suitcase with no place to keep anything useful for a hobby beyond a swimming costume or running gear - both solitary pursuits.
So, share your fears with DH. But I would say that he'd be reasonable to respond with "if you love me you'd be happy I finally have some people to socialise with and it concerns me that you're happy for me to work away and take the proceeds but not to trust me enough to keep it platonic"
After all, he's told you about the arrangement, not told you that he's watching TV with "Bob" rather than "Roberta".
Be aware, be alert, but don't be paranoid