Multi-Model and Assertion-Based Confidence Scoring to Improve Data Quality in Artificial Intelligence Training

Resumo

Artificial intelligence model performance fundamentally depends on training data quality, yet most real-world datasets contain noise, mislabels, and distribution shifts that compromise learning effectiveness. We present a general methodology that combines multi-model predictions with prompt-based confidence scoring to systematically extract high-quality training data from noisy datasets. Our framework operates through a multi-phase pipeline where outputs from multiple pre-trained models are evaluated using contrastive language-image prompts to generate normalized confidence scores for each sample. High-confidence samples are retained for training while low-confidence samples are filtered or queued for review. To demonstrate this approach, we apply the methodology to age estimation using facial images as a representative computer vision task. In our case study, outputs from two different architectures (Buffalo-L and a custom lightweight model) are processed through CLIP-based semantic evaluation with positive and negative age-related prompts. Samples scoring above empirically determined thresholds are used to train efficient custom models, achieving improved accuracy and computational efficiency. The threshold optimization process employs ROC curve analysis on validation data, while the pipeline integrates automatic data balancing and label correction mechanisms. This framework addresses the critical challenge of data quality in weakly supervised scenarios and provides a scalable approach adaptable to diverse domains including text analysis, speech recognition, and medical imaging. Our age estimation case study demonstrates 29% accuracy improvement and 40% training time reduction using only 32% of the original dataset, validating the effectiveness of quality-focused data selection over quantity-based approaches.