Although it's fine for them to do as they please and take the piss in the first place.
I have been making an effort lately to be more assertive, and have had counselling and read self help books.
Last Sunday I was invited to a friend's house for lunch. She told us to go round at midday. When I arrived not only was there no lunch, she hadn't even been shopping to get food for said lunch, and then said that it'd be nice for us to have the afternoon to chat and so she could get on with jobs, and we'd have an early tea instead. She also wanted me to look after her DC whilst she went to the shop to get food for tea, etc. I had only planned to go for a couple of hours and wanted to get back home to DH and my DCs, and so I said as she wasn't ready for lunch I'd stay and have a coffee and catch up and then head home, as I didn't want to be out all day. She wasn't very happy and moaned at me, and has ignored my calls and texts this week.
I have had other incidences where I have acted assertively when someone has been taking the pee and the person has taken offence at it, despite them being out of order themselves.