Start with your assessment types
List languages, subjects, descriptive answers, equations, diagrams, code, page formats and typical batch sizes. No single test result represents all of these conditions.
Choose two or three high-value workflows for a pilot rather than attempting an institution-wide rollout immediately.
Inspect the OCR and mapping workflow
Ask whether teachers can see the original page beside extracted text, correct OCR, identify missing pages and review question mapping.
A product that shows only the final mark makes errors hard to diagnose and moderate.
Evaluate rubric control
Faculty should be able to define question-level maxima, model answers, accepted alternatives and partial-credit rules. Check whether criteria can be versioned and whether changes are visible.
Test several valid solution methods to see whether the system handles alternatives fairly.
Require human control
Confirm who can approve, override and publish marks. Look for confidence flags, exception queues, random sampling and an audit trail.
For high-stakes use, align the workflow with examination policy and require qualified human approval.
Review data protection
Ask what student data is collected, where it is processed, how long it is retained, who can access it and how deletion works. Verify written contracts and security controls rather than relying only on a webpage claim.
Use anonymized or approved data during early testing.
Measure the pilot
Compare OCR quality, mapping accuracy, grading agreement, faculty correction rate, end-to-end time and support responsiveness.
Document failures and edge cases. A tool should be chosen because it performs acceptably on your papers, not because it has the highest generic score.
Check operational fit
Review batch limits, export formats, roster compatibility, user roles, training and support. Estimate total cost including scanning and faculty review.
Plan how the institution will continue when the service is unavailable or a batch fails.
Questions to ask every vendor
Ask for the exact workflow used when OCR is uncertain, the ability to export your data, retention and deletion terms, reviewer roles, model-change notifications and support for failed batches. Request evidence from comparable subjects and paper formats rather than an unrelated benchmark.
Confirm which capabilities are currently available and which are only roadmap items. Pricing should include scanning, storage, review and integration costs—not only a per-page model fee.
Make the final decision with evidence
Create a weighted scorecard using academic quality, faculty effort, privacy, reliability, support, reporting and total cost. Require stakeholders from teaching, examinations and IT to review the pilot. A slower product with transparent review controls may be a better institutional choice than a faster black-box score generator.