Enterprise AI Prompt Screening System

Abstract

Generative Artificial Intelligence (AI) tools are at present being routinely employed in business settings, but the prompts used are unstructured text that may contain personal data, financial information, credentials, or confidential details concerning an organization. There is also a risk associated with prompt injection, as this allows user instructions to bypass existing rules or expose the system's configuration. In most cases, there is no technical means of controlling access between the user and the model. The present paper proposes an Enterprise AI Prompt Screening System, which includes a deterministic and explainable screening stage before a large language model that is hosted locally. The system makes use of Spring Boot, Spring Security, Spring AI, React, and MySQL; it identifies sensitive entities by means of Java recognizers that operate on rules, detects signals that show that direct prompt injection is taking place, works out a weighted Prompt Risk Score, and then carries out one of five policy actions: Allow, Mask, Confirm, Review, or Block. Prompts that are merely allowed or sanitized are forwarded on to an offline Ollama model. The system is meant to be assessed by means of a synthetic labeled prompt dataset using the predefined acceptance criteria.