Vision AI Skills Checkoffs v2: A New Grading Methodology

Assessment systems improve when changes are versioned and measurable.

HealthTasks is formally upgrading Vision AI Skills Checkoffs from grading methodology v1 to v2.

Methodology v2 introduces a custom in-house video processing pipeline, decouples processing from the grading model, and records the methodology used for each submission. Programs get clearer evidence now and infrastructure that can improve over time.

What changes in v2

• Custom processing: HealthTasks has more control over how recordings become grading evidence.
• Model flexibility: Processing and grading are separate, so new models and approaches can be evaluated against the same rubric and evidence structure.
• Research-ready versioning: v1 and v2 can be compared without mixing results. Methodology tags can support research, study, and evidence-based improvements.

Early v2 results

In our initial internal comparison, methodology v2 produced:

• More specific descriptions of what was observed and what was missing
• Stronger rubric adherence and a stricter threshold for awarding credit
• The lowest observed rate of unsupported step ratings to date, with no unsupported step passes in the results reviewed so far

These are early internal findings, not a universal benchmark or a promise that automated grading can never make a mistake. Versioning lets us measure, document, and improve the methodology against real evidence.

The workflow stays familiar

Students still submit video against the program rubric. Vision AI still returns a first-pass grade with evidence, confidence, and timestamps. Educators still review exceptions and judgment calls. Methodology v2 makes that first pass clearer, more traceable, and easier to improve.

See Vision AI Skills Checkoffs

Learn more about AI Video Skills Checkoffs or book a demo to see how the workflow fits your program.