Detecting Hallucinations in Large Lanuage Model

Abstract

LLMs has created a revolution in Natural Language Processing, it enables new capabilities in text synthesis, question answering, summarizing facts, and conversational AI outputs. However, This models are exceptionally amazing still they may often generate hallucinations, i.e. the output is inaccurate and not supported by solid facts or informations. Hallucinations are widely affecting the performance of LLMs making it unreliable and limit their applications in Various sectors like banking, Healthcare, cybersecurity and legal services where the fact based assessment and synthesis is highyly important aspect.

In this study, we are deep diving in the topic of Hallucinations of LLMs, covering causes of hallucinations, varied forms of hallucinations, detection of approaches, standard assessment metrics, and the current issues in hallucinated outputs. The core is focused on working a Hybrid Hallucination Detecttion Framework (HHDF) that uses retrieval-based verifications, semantic analysis, lexical analysis, confidence estimation and self learning based model to understand the key fundamentals of hallucinations and responses. We are more focused on calculating a confidence score on each responses to detect the hallucinations and help us to gain more precise and trustworthy outputs.

We are more focused on implementing approaches on public benchmark datasets like HaluEval and Truthful QA and measure the performance in terms of accuracy, precision, recall, F1 scores, and hallucination detection rates. The methodology helps us to give a more reliable and interpretable LLM response by using several verification procedures and minimizing factual inaccuracies.