Difficult to know without a google maps view of the roads…
but worth noting that in British law there is no such thing for drivers as Right of Way… the only time it’s mentioned in the Highway Code is to specifically state that it doesn’t exist.
it is really important to understand this as a driver as it helps in how you interact with others… in many situations you might have Priority but that only means you have first dibs if you are both there at the same time - so, a roundabout, you have priority if coming from the right, a road where cars are parked on the other side forcing traffic into your lane - you have priority… a junction like this you have priority…
but only if you are both there at the same time - if the other car is there first they are allowed to use the road / junction / roundabout, and in doing so before you arrive they gain priority… the fact that you arrive on the scene after they have started their manoeuvre doesn’t mean that you have priority, or right of way - it only applies when you both arrive at the same time and a decision has to be made as to who goes first…
the Highway Code also states that it is your duty to avoid an accident whatever the other person is doing - even if they are in breach of the Highway Code, you must give way rather than have an accident…
If all drivers obeyed the Highway Code and drove with more humility and less arrogance (believing that they have a right of way) we would have far fewer issues / incidents/ accidents on the road…
so back to your situation - yes if you are both there at the same time you have priority, but if they are ahead of you they can cross and then they gain priority and you have to give way… it is very subtle and down to timings and whether they cut you up / you could see them crossing and stormed up to them / etc…