Before I'm accused of being PFB, I have no problem with people holding my baby, it's the passing around and not giving the baby back when they need feeding, settling or just a break from not being held that annoys me.
To avoid drip feed, this has happened during extended family visits. When I have asked for baby back to feed or settle (settling being the main issue as baby doesn't sleep during these holding marathons), I am met with 'can I have a hold before you feed them' or 'can I try settling them', which I am mostly firm on, but not always as paranoid about coming across difficult. Meanwhile I'm on edge watching my baby showing signs of hunger or overstimulation, the latter obviously worsening throughout the day.
AIBU and if not, any advice for managing this without falling out with anyone?