THE SYNTHETIC DATA CONTAMINATION INDEX (SDCI): A Measurement Framework for Quantifying Recursive Contamination and Preventing Model Collapse in Generative AI Systems

Abstract

Abstract Generative artificial intelligence systems increasingly rely on large-scale datasets that may contain substantial volumes of synthetic or machine-generated content. Research has demonstrated that recursive training on synthetic outputs can degrade diversity, distort statistical distributions, and contribute to model collapse. The Synthetic Data Contamination Index (SDCI) provides a standardized, model-agnostic framework for quantifying contamination risk within training corpora before model training begins. The index evaluates five measurable dimensions: synthetic ratio, recursive generation depth, provenance confidence, linguistic homogenization, and human anchor deficit. These variables are combined into a single normalized score (0-100), enabling direct comparison across datasets and supporting pre-training governance decision Introduction The rapid expansion of generative artificial intelligence has introduced a structural risk in training data pipelines. When models are trained on outputs produced by other models, recursive contamination may emerge. This process gradually reduces statistical diversity, removes rare patterns from distributions, and can ultimately lead to model collapse. Keywords Artificial Intelligence Governance, Synthetic Data Contamination, Model Collapse, Training Data Integrity, Dataset Governance, Machine Learning Risk Metrics, Provenance, Human Anchor Data, AI Safety.

Ähnliche Arbeiten