There are so many questions around your post, to try and understand what the real situation is. So difficult to work out from posts. For example
Who decides how much work you have to do? How is your part time working taken into account in deciding what needs to be done?
Is everything in the pipeline of equal urgency ?
How often do you genuinely not manage to get through everything in the pipeline during your working hours?
What come back is there for you, if you do not manage to finish everything?
Is there another member of the team who can deal with anything urgent, which cannot wait until you are next working?
Do you ever find yourself without anything in the pipeline so that someone else can off load into you ?
What do you think your manager would say, if they knew how anxious you feel in these situations?
Can you talk about these feelings to any other colleagues ? Are you the only person with part time hours. Do you feel there is an us and then situation between pt snd ft people ?