I think the thing is there is no right or wrong answer to this. Because catagories like white, black, and mixed race are to a large extent socially defined and depend in part, but not entirely, on how you are seen by others.
Ethnically you are Scottish Jamaican, that's a fairly objective measure.
Culturally, you probably to a large extent share the culture of your adopted family, unless you've had some other cultural influences.
As far as how people treat you, as "white" or "mixed" or whatever, that will depend on how they see you. For some people, no one will be in doubt, for some people, they will. It might depend on if they see you with family orin a particular setting, or the background of the person looking might make a difference.
Then, if you adopted family had some special advantages, like being super rich or members of the aristocracy, you might have all kinds of benefits based on that, which someone else would not have.
I think you can call yourself what you want, really, and your friend's comment seems odd, but it's also true that other people may or may not see you as black or mixed race.