There are so many phrases that you can pick apart and show they make no literal sense. IMO the only thing that really matters is that people know what you mean. And saying 'I don't believe in paperclips' has the benefit of being marginally less boring than 'I don't like to use paperclips', though, in the context of a conversation about paperclips, perhaps this is not a distinction visible to the naked eye.