Yes, I have really had my eyes opened about AI recently. Touted as this great helpful amazing thing, but in fact so unbelievably and frighteningly flawed as a tool. I was at some teaching about it recently where I was alerted to the phenomenon of AI hallucination. The teacher showed us how input asking about legal cases on a certain issue resulted in completely fabricated cases being presented in a completely convincing way.
So I tried asking an AI engine about whether there are any legal cases about me (I have a completely unique name which no-one else in the world has, if you put my exact spelling and my title, so if it threw up a case it could only be linked to me, not someone else with the same name).
It turns out that, depending on when you ask, I am either one half of a lesbian couple in South Africa who had a child by artificial insemination and fought a custody battle with my ex-partner (I won that case, it seems), or an Australian anaesthetist who left a patient brain damaged through malpractice (I lost that one). The only two nuggets of truth in there are that I once lived in South Africa (not for decades and not at the time the case supposedly occurred), and I am a doctor (not an anaesthetist, nor have I ever caused anyone brain damage).
That little exercise stopped dead in their tracks any nascent ideas I had of using AI for anything other than writing amusing poems about charismatic sheep, or producing pictures of ballet dancing guinea pigs...