I recently read a very interesting paper (https://arxiv.org/pdf/2401.01313v1) titled "Why Language Models Hallucinate" by Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala, and Edwin Zhang. The authors analyze the causes of so-called "hallucinations," which are situations where a language model (LLM, or popularly, AI) says something that sounds absolutely credible but is completely untrue. Have you ever asked an AI chatbot a question and received a beautifully phrased, confident... falsehood? It might be a made-up book title, a non-existent historical fact, or, as in one study's example, three different, incorrect birth dates for the same person. This phenomenon, known in the industry as "hallucination," is one of the biggest barriers to fully trusting artificial intelligence. A new scientific paper sheds light on this problem, arguing that hallucinations aren't a mysterious glitch but a logical consequence of how we train and evaluate language models. In short: we ourselves have taught AI that guessing pays off. The Original Sin of AI: Errors from the Training Stage It all begins at the "pretraining" stage, when the model digests vast amounts of text from the internet to learn language patterns. The study's authors show that even with perfectly clean training data, statistics are relentless. They explain this with a clever comparison to a binary classification problem. Imagine the AI's task isn't to generate text, but to answer "true" or "false" to statements. It turns out that generating correct sentences is significantly harder than simply evaluating their correctness. What's more, the researchers established a mathematical relationship: The error rate of a model's generated output is at least twice as high as its error rate in judging what is true and what is false. This is especially evident with facts that appear very rarely in the training data. If information about someone's birth date appeared only once across the entire internet, the model statistically...