If your library is a bus ride away, it costs too much.(there is no library where my ex lives) my local library is too far to walk when I am being too disabled. or DD is.
My local library is too busy for my autistic children to access at the weekend. Life is tough enough without dealing with autistic meltdowns in public. It is soul destroying.
If you are working all hours God sends to make ends meet there is no bloody time or energy. Nor if you are disabled. Being disabled often makes you poorer than you otherwise would have been.
If you struggle to read yourself you are not going to want to go. Or if English is not your first language.
If your library is full of people who are not like you you may feel out of place.
Getting the buggy down several flights of stairs as the lift has broken. Or there is no lift. (You have to carry baby and buggy and bag and hold onto a toddler down several flights of stairs) and worse, back up again.
Don't want to navigate the dog shit and vomit and broken glass (regular on my route to town) with the buggy when you have nowhere to clean the bloody thing if you live in a flat.
Not going out in the dark and needing to be home before dark. (Definitely don't go out after dark if can help it during October November)
There are lots of reasons why people might not want to/ can't get to the library.
It shows lack of imagination and a certain middle class judginess to not understand there are lots of barriers to accessing libraries, museums etc that you don't know about.