I belong to a reading group - and some of the reservations you have do apply, but I think the benefits probably outweigh them. - Sometimes the book we read is not to my taste, but more often I have discovered authors or books that I really enjoy, that I would probably never have tried if they hadn't been the 'set book'.
We meet once a month in a small meeting room in a local hotel (they let us have the room for free as long as we all buy coffees). We choose the books in several different ways - we look at lists of 'book group reads' (usually available from libraries or online if you google) or often our library has a theme going such as 'summer reads' or 'books which inspired films' etc - so we pick something from those lists. Or one or other of us suggest something we like the look of - we try to mix it up - classics (modern and traditional) modern novels, locally set novels, detective fiction, the odd biography, although we are mostly fiction.
We get our books from the library - so we are restricted to what they have available in reasonable numbers, - but if you are buying the books yourselves you'd have more freedom.
How you organize it is really up to you - but we go round the group and everyone says what they thought of the novel, if and why they enjoyed/didn't enjoy it, then we all discuss it in more general terms, and talk about some of the wider themes of the book - it's pretty informal.
Just to give you an idea, some of the books we;ve read over the last year or so, - The Bell Jar (SYlvia Plath) Started Early Took my Dog (Kate Atkinson) The Blue Book (A L Kennedy) The Way We Live Now (Trollope) Beloved (Toni Morrison) and we read Dawn French's autobiog. - Next month we're doing Iain Banks - The Crow Road.
Quite a lot of book clubs specialise in a particular type of fiction - we are a bit less restricted in our choices.
I think there is info on Google about setting up book clubs -which might be helpful.
We sort of run ours in a co-operative type way - one person orders the books from the library, I e-mail everyone a reminder the weekend before the meeting, - everyone has a little job they do to keep it going.
We do often disagree about the book, - it's quite interesting how differently people can feel about the same novel, - and we have some animated discussions - but differing opinions are good to hear and its all very friendly 