Question

A document-extraction tool uses ML models to extract invoice fields (vendor, amount, date). The models return confidence scores (0.0–1.0) for each extracted field. In production, you observe: 1. The agent proceeds with low-confidence extractions that are incorrect 23% of the time. 2. The agent requests unnecessary human review for 31% of extractions that were actually correct. How should the tool’s output be restructured?

Accepted Answer

B. Return fields with confidence scores, plus a request_review boolean computed using your tested confidence thresholds, along with a review_reasons array explaining which fields triggered review.

Answer

Return fields with their raw confidence scores and add detailed few-shot examples to your system prompt demonstrating how to interpret different confidence ranges and when to request human review.

Answer

Compute an aggregate extraction_quality score across all fields and return it alongside the extracted values. Include a text summary describing the overall extraction reliability.

Answer

Return fields organized into verified and needs_verification objects based on confidence thresholds.