I have done that job and it was the hardest, most stressful job I've ever done. My advice is to keep it and try to be the best you can be at it so you can use it as a reference for another non-receptionist office job.
I found 90% of it was outward appearance - not just your look but confidence in how you handled the job. You don't have to be a beauty queen but I found if you looked extremely professional and acted confident and very professional you got bullied less. I'm not saying I was the confident one, I wasn't, I was the one that got bullied.
It also helps at the same time you are very professional to be bubbly and helpful to all employees and visitors. I found I played psychiatrist to most of the employee and while on break they would come up to me, to 'shoot the shite' with me while I was trying to work, etc. They would drape their arms over my desk top and it so felt like a invasion of my personal space! Ug - it was very hard as I was and still am not much of a people person.
I know there are courses (one day) you can take on your career and I think that can be extremely helpful. I know everyone hates you being gone because then they have to cover the phones don't they? You are the lowest on the totem pole but if you are out it's as if the office goes into emergency mode.
If you can't do a course research on the internet how to present yourself professionally and implement it asap. Do a complete change - attitude, clothing, etc. if needed to make you at least appear super professional (I'm not saying you aren't!) and confidence will grow for you. People give you more slack if you appear super confident in this role and then if you get behind they will be more understanding. All this professionalism talk is because you represent your office. If you provide a 'sleek package' (sorry but it's true) the office staff takes pride in you because you represent them, this in turn makes them treat you better and cuts you more slack. I doubt you are rubbish at your job, I'll bet more it's that your fellow employees are more than demanding so you need them on your side.
I was the little bookish mouse at the receptionist desk and was treated like crap although I tried hard and churned out good work. The woman after me was super professional, bubbly, etc. and dressed the part and the employees expected less of her, treated her so much better, and saw her as a representative of them. I'm not saying it was a fantasy job for her, but it made it endurable for her until, she too moved on.