Yes we would be forced to as employers would just change people's hours using the "needs of the business" clause. Pretty much every major retailer has this clause in their contracts to justify changing days/hours employees have to work and if they objected then they'd just lose the hours or be given other inconvenient hours (or in the case of temps/zero hours workers just wouldn't be scheduled again) which will essentially force people to work Sundays or be left worse off.
I have known people be told they need to do a 6hr shift on a particular day and when they refused, the manager then offered a split shift of 2×3hr blocks as that was "the only option to keep their hours as there's no other hours available" ... The staff member ended up agreeing to the 6hr shift because they couldn't afford to lose the hours but the split shift just wasn't practical due to their circumstances. So whilst nobody held a gun to their head to make them take the hours, the manager knew which buttons to push to force the hours they wanted through.
I took a job knowing that my hours would be any time between 6am and 11 pm Monday to Saturday and 11am to 5pm Sunday. So I knew that, no matter what days/hours I was scheduled and whether or not my days off were the same every week, every week I was guaranteed to always have a lie in on a Sunday morning and also be guaranteed to be home to have dinner and spend the evening with my DD.
If stores were not restricted on Sundays that guarantee is gone and the hours I am expecting to work (and that I signed up to work) are changed. So the fact I took the job, knowing the hours, means nothing because they can just be changed unilaterally and I have no say on the matter.
Nobody I work with want more hours with customers on the stores, especially on a Sunday as that's the day we can get the most work done as the store is closed longer, and if you want "bonus hours" then there are always overtime shifts for stocking/home shopping picking etc that are jobs done when the store is closed. Additionally things like annual stock taking, inventory management, display changes, maintenance to machinery/fixtures etc are all scheduled, as much as possible, on Sundays as it gives the teams the longest time possible without customers in the building, which makes it safer and less inconvenience for everyone.