It's understandable OP that you feel it's harsh, as I know you didn't miss it because you couldn't be arsed/didn't care. (In fact, most people probably don't do that!) But it costs some twenty quid or so for each appointment missed, and it's very frustrating for the surgery who (probably) has their own budget, and struggles anyway.
A very good pal of mine is a doctor and he says the head doctor of the surgery pays for and has to budget for EVERYTHING. Drugs, specialist care (physio/mental health care etc,) rent for the building, bills, salaries for receptionists, doctors, practice nurses, all supplies like furniture and stationary and computers etc for the practice. Look at ANYthing in your surgery next time you go, ANY item, and the person/head doctor who runs the surgery has had to account for that in his budget. And he also has to budget for appointment times.
So I can understand your doctor getting snitty. It's an annoyance at best when people miss appointments, and a financial burden at worst.
Our head doctor gets frustrated because he finds it almost impossible to ban people though, as we are the only surgery for 15 miles. (It's a rural area.) And some people cannot drive. He said he is tempted to introduce a 'charge a tenner when you miss an appointment' system. I bet a lot less appointments would be missed then!
PLEASE try not to miss anymore.