Competency and Evaluation Goals

Up: Whitelabel Training Program · Practice: HVE Sample Labs · Audience: Learner Personas

Program Goal

Learners should deliver or evaluate a bounded piece of AI-assisted engineering work with enough evidence that another person can understand the decisions, verify the outcome, and continue responsibly. Tool fluency supports this goal; command memorization and attractive generated output do not establish it.

This is an initial Frontier evaluation design supporting the Microsoft-issued individual certificate recorded in Certification and Assessment Design. The rubric and pass thresholds still need approval and calibration; these samples alone do not establish full program competence.

Common Competency Model

Competency	Observable behavior	Acceptable evidence
Problem framing	States the user need, boundaries, and checkable acceptance criteria	Brief with a resolved ambiguity or explicitly owned question
Evidence and context	Finds relevant facts, distinguishes assumptions, and reuses adequate evidence	File/source references and a focused handoff context
Planning and decisions	Sequences bounded work and identifies approval needs	Plan linked to requirements and decision rationale
Execution or role contribution	Produces the change or role-specific deliverable within scope	Diff, data output, clarified requirement, or review record
Validation and acceptance	Checks behavior and challenges unsupported completion claims	Actual check results plus requirement-level acceptance
Collaboration and handoff	Identifies unresolved work and a next owner	Resumable artifact packet and explicit disposition

Proposed Rating Scale

• Not observed: no sufficient opportunity or evidence; do not treat as failure or success.
• 1 — Guided: completes the behavior with step-by-step assistance.
• 2 — Independent: completes a bounded task and explains the evidence with ordinary reference material.
• 3 — Adaptive: handles ambiguity or a changed condition and explains the trade-offs.

Record assistance used, model/tool failures, and whether the learner authored, inspected, or merely received each artifact. AI assistance is expected. Ask the learner to explain one decision and handle a small change to check understanding.

Role-Specific Goals

Route	Independent goal	Additional emphasis
Technical Explorer	Explain acceptance and make a useful evidence-based handoff	Demonstrate a check with guided coding support
Application Practitioner	Implement and validate a bounded requirement	Edge cases, regression awareness, scope control
Data and AI Practitioner	Produce correct, reproducible results under a stated data contract	Invalid inputs, aggregation semantics, data provenance
Technical Lead / Architect	Evaluate a change and justify a consequential decision	Standards, trade-offs, unresolved risks
Delivery / Product Lead	Clarify scope and record a defensible acceptance decision	Dependencies, decisions, ownership, client handover

Code correctness is required for an engineering completion claim. A product lead can demonstrate competence through requirements and acceptance work without independently implementing code. Record role and observed competencies explicitly.

Evaluation Sequence

1. Placement: use the short activity in Learner Personas; establish support needs.
2. Formative practice: coach during one sample; record misconceptions, not a credential outcome.
3. Individual demonstration: use a small variation after practice, with references and AI assistance allowed.
4. Evidence review: learner explains a decision, shows validation, and identifies the next owner.
5. Feedback and retry: name the missing evidence or behavior, offer another bounded task, and retain both observations.

Provisional Pilot Completion Rule

For pilot learning feedback only, propose a rating of at least 2 on all six common competencies, assessed through the learner’s route. Any unobserved competency needs another opportunity. An unsupported acceptance claim or unresolved correctness defect prevents declaring the affected outcome achieved, regardless of an average score. Final certification rules require program approval and pilot calibration.

Review is deliberately required in our assessed samples even though Microsoft permits optional RPI acceptance review. See HVE RPI Workflow.

Assessor Calibration and Fairness

Have two instructors independently score the same sample packet before the pilot, compare disagreements, and add evidence examples. Evaluate decisions and results rather than exact prompt wording or similarity to an instructor solution. Give equivalent tasks and reasonable accessibility accommodations. Distinguish environment failure from demonstrated skill gaps.

Observation Record

Use one record per learner demonstration:

Field	Record
Learner identifier, date, route
Sample and variant; environment/source baseline
Requirement and artifact references
Ratings for the six competencies
Assistance and tool limitations
Check results and acceptance disposition
Learner explanation / changed-condition response
Assessor, feedback, next practice

Pilot Questions

• Do the tasks distinguish independent judgment from copying generated output?
• Are comparable ratings achievable across role routes?
• Which criteria cause assessor disagreement?
• How much time and assistance does each route require?
• What does Microsoft or the brand owner require for credential claims?

Use results to revise this rubric before adopting pass thresholds. Retention and access for learner evidence belong in Environment and Setup Plan.