I need advice please 
Basically I've been working for the same people for 3 years. Things have always been good, however I had a run in with a co-worker in July. Things got heated and outside of work we had words. The next morning I Called her to sort it out, we both apologised and all was good. Later that day I got a call from my boss saying I was being moved to another location, I asked why and he said nothing to do with run in with co worker, it was pre planned. Spoke to co worker about it in passing a few days later, she said 'oh I asked him to do that after our falling out' I moved for a week and he moved me back. I lost some hours but all was grand.
Said co worker has handed in her notice as she has another job in her field that's better for her. When she told me I hugged her and congratulated her, all was fine. Got a call on Friday from boss saying I'm being moved permanently to the other location and he is pulling someone from there and swapping us basically. I expressed my concerns as 1. My hours would be cut significantly (10hrs+) and it's more difficult for me to get home from this shop. Boss went off on one, said I can get two buses it's no big deal and if I don't move he's going to cut my hours anyways. In the place he wants to move me I would be always working alone, I told him this concerned me as I struggle to do heavy lifting due to arthritis. He called me a bull shitter and a useless worker. I'm a neither. I work very hard for my boss and always have and he knows that. I was very distressed about all of this and the next day I had a panic attack.
Can anyone help me? What can I do? As it stands I have told him that if he is making me move then I will but I really don't want to, he ended the phone call with 'I've a really fuckingn busy day, you've pissed me off and now I'm shouting so I'm away' and hung up on me. I'm so stressed and I don't know what to do. I think he is moving me to punish me for the other girl leaving, as he has slept with that girl a few times so she gets better treatment than the rest of us. My boss is also my DP friend, they play golf together. Boss brought up the convo to Dp by saying 'your girl is fucking nuts' my partner obviously stuck up for me but tried to stay out of it. Boss made jokes about it and said he's raging co worker is leaving because she is a 'dirty bitch in bed'. He is so inappropriate. Also to avoid drip feeding he is twice the age of all his employees, and has stepped out of line on a few occasions e.g he smacked my back side when I was working and also stripped down naked infront of another girl who works in the place he wants me to move to. She was mortified. I can't move location as it will cost me money on getting home, I'll lose hours and I'll be in the shop alone with his and he's a creep.
Advice please
Sorry this is all over the place, I'm trying to avoid a drip feed