To me it sounds like she was, for whatever reason, particularly alert for anything that could be perceived as a slight, and reacted accordingly. I mean, it was in addition to other gifts, and as you say, she doesn't have a general aversion to used books.
If it weren't for the fact she's your mother, I'd wonder if this was a clash of expectations and perceived implications — like if one person came from a family where thoughtfully-chosen second-hand books were a normal present and the fact of a book being second-hand couldn't possibly carry any unspoken meaning, while the other person came from a family where books as gifts were always new, where part of the present was getting a brand new untouched book all for yourself, and giving someone a second-hand book as an actual gift would be so unusual it must be sending some kind of message. But closely-related family? I can't see there being any of that kind of misunderstanding or culture clash — even if you'd developed different attitudes, I'd guess you'd probably know these things about each other. Unless you've had very little contact for a long time, I suppose.
I do kind of wonder whether waiting until she was home to tell you how she felt was because she didn't want to spoil the atmosphere at the get-together, or whether it wasn't until later (after some brooding and analysis) that she decided it was disgusting of you to give her a secondhand book.
But in general, when it comes to second-hand books as gifts, I think it depends on a lot of variables TBH.
- Yes: if in 100% brand new condition, never been opened (or at least no more than a book bought from a bookshop), smells new, makes the right noise when you open it, fully intact dustcover (if applicable), no sunlight damage, no dust, not a mark on it inside or out, they'd never know it wasn't new if you didn't tell them, etc.
- Yes: if it's something difficult to find or out of print, or an older edition chosen for the aesthetics of the cover/binding/textual differences/illustration differences/signed/first edition/special to that person for some reason — anything like that where it would be impossible to buy new, and you know they'd appreciate that specific aspect of it over a new one.
- Yes: if it's a more casual gift (e.g. not wrapped, more "Hey I just finished this and think you'd love it/saw this in the charity shop and thought of you/got you a copy of that book I was telling you about") and in reasonable condition (i.e. has all its pages, no mysterious gross staining or spider poo).
- Yes: if it's someone you know doesn't care in the slightest whether their books are new or secondhand, and might even be annoyed you wasted your money on a new one. But even then, if it was a special birthday or something I might feel uncomfortable about that.
- Yes: if it's a secondary gift in addition to the main one, just something extra to unwrap as a bonus. But only if I knew they were okay with second-hand books in general, and it would probably be a book with a specific reason behind it. And I might refer to the main gift as their "real present" if I had any concerns that they might think I was sending a message by giving a second-hand book.
- Maybe: if you know each other well, both of you know finances are tight for you, there's some thought gone into the book chosen, and the options are either 1. you give a reasonable-condition second-hand copy of the book you want to give 2. you go without something else to buy them a new copy, or 3. no book at all, some cheaper gift instead that's in budget. It depends on a lot of things… personal sensitivities/family culture/closeness of relationship/expectations and so on. I might not risk this personally, and maybe go for option #3 instead, unless I knew that particular recipient would take it in the spirit of generosity it was given in, and actually be glad that I hadn't spent more on them than I could afford.
- No: the newest release in their favourite author's series that they collect and display in a row on their bookcase; any book that's designed primarily to be an attractive gift or coffee table book; stuff that's easily-obtainable as a cheap new paperback that I can comfortably afford; a random book picked up just because it was cheap in the charity shop; a gift for a big birthday/some other significant event; situations where you've bought an equivalently-related person (e.g. your brother vs your sister) something obviously new and more expensive (and none of the out-of-print/aesthetic/first edition/special to the person etc. exceptions apply); a recipient who I thought could possibly be upset by, take offence at, or twist the meaning of being given a second-hand book; a recipient who's squeamish about second-hand books and library books in general, and doesn't buy or read them. (Well, unless it was a perfect "they'll never know if you don't tell them" specimen. Anything goes in that case 😈)
I suppose some of the big reasons we give people gifts (other than habit/expectation) are that we want to make them happy, to show them that we know them and care about them, to build or maintain relationships, to stay on the right side of someone, maybe even to create reciprocal ties of obligation. Probably lots of other reasons too. Hopefully it's mostly that first one, making people happy.
With all the possible reasons to give a gift, the effect on that person is what matters to me. It doesn't matter if I think it's silly to be freaked out at the idea of other people's hand-germs on a book, or if I genuinely don't mean to insult or demean someone by giving them a second-hand book. If I think there's a chance that the recipient will interpret the underlying message of my gift as less "I care about you and thought that this present would make you happy" and more "I don't think you're worth the expense of a new book/I've given you something used and dirty because I don't care that it makes you uncomfortable/whatever", and therefore it won't make them happy or help build/maintain our relationship, then I won't give that gift, because it probably won't achieve what I want it to, whether I personally feel they should just be grateful or not.
But if a recipient is determined to find SOMETHING about your gift to take umbrage at, then they'll find something no matter how thoughtful you are.