Scope
EvalAI is an AI-assisted evaluation platform. It helps extract handwritten responses, organize answers, generate rubric-based score suggestions and prepare reports. Authorized educators remain responsible for final academic decisions.
Processing stages
The documented flow is PDF-to-image conversion, conservative image preparation, OCR extraction, confidence and region checks, question-answer mapping, model-answer or rubric comparison, faculty review, mark approval and report export. Each stage has a different error profile and should be measured separately.
Validation dimensions
Evaluate page completeness, OCR word or character quality, question-mapping accuracy, agreement with independent faculty marks, faculty correction rate, exception detection, turnaround time and critical-error rate. Do not combine these measures into one unsupported accuracy claim.
Representative data
A pilot set should include multiple handwriting styles, clean and difficult scans, long and short answers, corrections, formulas, diagrams and subject-specific formats. Personal information should be minimized and handled under institutional policy.
Human review
Review is required for low-confidence extraction, missing pages, uncertain question mapping, score boundaries and unusual answers. High-stakes assessments should follow institutional policy and may require full review of every suggested score.
Change control
Revalidate when the OCR engine, AI model, preprocessing, rubric, scanner or paper format changes. Record the software and rubric version used for each pilot or production run.
Limitations
Performance varies with handwriting, scan quality, language, subject and rubric clarity. Mathematical notation, complex diagrams, mixed scripts and handwritten code can need extra review. AI output can be incomplete or incorrect even when it appears confident.
Publication of results
Public benchmark claims should include test-set description, sample size, metrics, exclusions and review policy. EvalAI should publish only measured results that can be reproduced or independently examined.