# 5. Target Exam Analytics — Test Cases

User Type: **Super Administrator**
Source: *Mi Digital Academy - Education CRM Features Document*
Spec: target_exam_analytics.md — every feature, sub-feature, and rule covered

## Test Execution Policy

- Zero tolerance: any deviation from the documented behavior is a defect.
- Every failed test is logged with a Bug ID, the feature, the sub-feature, the expected vs actual result, and the severity; 100% of bugs are fixed before the group passes.
- 100% pass rate is required for the group to be marked complete.

## Coverage Matrix

| Feature | Sub-feature / Rule | Test IDs |
|---------|--------------------|----------|
| 5.1 Prediction Accuracy Analytics | Predicted vs actual score comparison | TC-SA-TE-40-001 |
| 5.1 Prediction Accuracy Analytics | MAE and bias metrics | TC-SA-TE-40-002 |
| 5.1 Prediction Accuracy Analytics | Accuracy by exam type and cohort | TC-SA-TE-40-003 |
| 5.1 Prediction Accuracy Analytics | Calibration analysis | TC-SA-TE-40-004 |
| 5.1 Prediction Accuracy Analytics | Rule: no actual result → excluded | TC-SA-TE-40-005 |
| 5.1 Prediction Accuracy Analytics | Rule: bias of zero → no systematic error | TC-SA-TE-40-006 |
| 5.1 Prediction Accuracy Analytics | Rule: below minimum exams → "insufficient data" | TC-SA-TE-40-007 |
| 5.1 Prediction Accuracy Analytics | Rule: never estimated for missing actuals | TC-SA-TE-40-008 |
| 5.1 Prediction Accuracy Analytics | Audit logging of prediction accuracy analytics | TC-SA-TE-40-009 |
| 5.2 Plan Adherence Analytics | Adherence rate (on-time completion %) | TC-SA-TE-41-001 |
| 5.2 Plan Adherence Analytics | Slippage analysis (average days) | TC-SA-TE-41-002 |
| 5.2 Plan Adherence Analytics | Adherence by exam type, cohort, horizon | TC-SA-TE-41-003 |
| 5.2 Plan Adherence Analytics | At-risk and off-track identification | TC-SA-TE-41-004 |
| 5.2 Plan Adherence Analytics | Rule: no completed tasks → 0% not excluded | TC-SA-TE-41-005 |
| 5.2 Plan Adherence Analytics | Rule: slippage over completed tasks only | TC-SA-TE-41-006 |
| 5.2 Plan Adherence Analytics | Rule: at-risk = at threshold not off-track | TC-SA-TE-41-007 |
| 5.2 Plan Adherence Analytics | Rule: no active plans → "no data" | TC-SA-TE-41-008 |
| 5.2 Plan Adherence Analytics | Audit logging of plan adherence analytics | TC-SA-TE-41-009 |
| 5.3 Mock Test Trend Analytics | Score trend (per student across mocks) | TC-SA-TE-42-001 |
| 5.3 Mock Test Trend Analytics | Improvement rate (score delta) | TC-SA-TE-42-002 |
| 5.3 Mock Test Trend Analytics | Trend by exam type and cohort | TC-SA-TE-42-003 |
| 5.3 Mock Test Trend Analytics | Plateau detection (no improvement over N) | TC-SA-TE-42-004 |
| 5.3 Mock Test Trend Analytics | Rule: fewer than two mocks → no trend | TC-SA-TE-42-005 |
| 5.3 Mock Test Trend Analytics | Rule: plateau after configured N mocks | TC-SA-TE-42-006 |
| 5.3 Mock Test Trend Analytics | Rule: delta on comparable mocks only | TC-SA-TE-42-007 |
| 5.3 Mock Test Trend Analytics | Rule: no mock data → "no data" | TC-SA-TE-42-008 |
| 5.3 Mock Test Trend Analytics | Audit logging of mock test trend analytics | TC-SA-TE-42-009 |
| 5.4 Gap Closure Analytics | Gap identification (weak topics) | TC-SA-TE-43-001 |
| 5.4 Gap Closure Analytics | Gap closure rate | TC-SA-TE-43-002 |
| 5.4 Gap Closure Analytics | Gap persistence | TC-SA-TE-43-003 |
| 5.4 Gap Closure Analytics | Gap closure by exam type and cohort | TC-SA-TE-43-004 |
| 5.4 Gap Closure Analytics | Rule: closed only at target threshold | TC-SA-TE-43-005 |
| 5.4 Gap Closure Analytics | Rule: persistent gap flagged for intervention | TC-SA-TE-43-006 |
| 5.4 Gap Closure Analytics | Rule: no gaps → "no gaps" | TC-SA-TE-43-007 |
| 5.4 Gap Closure Analytics | Rule: insufficient attempts → not counted | TC-SA-TE-43-008 |
| 5.4 Gap Closure Analytics | Audit logging of gap closure analytics | TC-SA-TE-43-009 |
| 5.5 Cohort & Institution-Level Reporting | Cohort report (aggregated metrics) | TC-SA-TE-44-001 |
| 5.5 Cohort & Institution-Level Reporting | Institution report (comparable cohorts) | TC-SA-TE-44-002 |
| 5.5 Cohort & Institution-Level Reporting | Report export (PDF, CSV) | TC-SA-TE-44-003 |
| 5.5 Cohort & Institution-Level Reporting | Report scheduling (periodic delivery) | TC-SA-TE-44-004 |
| 5.5 Cohort & Institution-Level Reporting | Rule: no students → "no data" not empty chart | TC-SA-TE-44-005 |
| 5.5 Cohort & Institution-Level Reporting | Rule: only comparable exam types compared | TC-SA-TE-44-006 |
| 5.5 Cohort & Institution-Level Reporting | Rule: no data → header-only export | TC-SA-TE-44-007 |
| 5.5 Cohort & Institution-Level Reporting | Rule: scheduled delivery respects email prefs | TC-SA-TE-44-008 |
| 5.5 Cohort & Institution-Level Reporting | Audit logging of cohort & institution-level reporting | TC-SA-TE-44-009 |

## 5.1 Prediction Accuracy Analytics

### TC-SA-TE-40-001 — Predicted vs actual score comparison
**Type:** Positive
**Covers:** 5.1 → Predicted vs actual
**Preconditions:** A Super Admin account is active; students have predictions and actual results.
**Steps:**
1. As a Super Admin, open Target Exam Management → Analytics → Prediction Accuracy.
2. Review the predicted vs actual score comparison per student and exam.
**Expected Result:** Predicted vs actual score comparison — delivered exactly as documented.
**Priority:** Critical

### TC-SA-TE-40-002 — MAE and bias metrics
**Type:** Positive
**Covers:** 5.1 → MAE and bias
**Preconditions:** Predictions and actual results exist.
**Steps:**
1. Inspect the mean absolute error (MAE) and bias metrics.
2. Verify the metrics are computed correctly.
**Expected Result:** MAE and bias metrics — delivered exactly as documented.
**Priority:** High

### TC-SA-TE-40-003 — Accuracy by exam type and cohort
**Type:** Positive
**Covers:** 5.1 → Accuracy by exam type/cohort
**Preconditions:** Predictions and actual results exist.
**Steps:**
1. Slice the accuracy by exam type and cohort.
2. Verify the sliced metrics are correct.
**Expected Result:** Accuracy by exam type and cohort — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-40-004 — Calibration analysis
**Type:** Positive
**Covers:** 5.1 → Calibration analysis
**Preconditions:** Predictions with confidence exist.
**Steps:**
1. Review the calibration analysis (predicted confidence vs observed accuracy).
2. Verify the calibration is displayed correctly.
**Expected Result:** Calibration analysis — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-40-005 — Rule: no actual result → excluded
**Type:** Edge
**Covers:** 5.1 → Rule: excluded
**Preconditions:** A student has a prediction but no actual result.
**Steps:**
1. Review the accuracy metrics.
2. Verify the student is excluded from the accuracy metrics.
**Expected Result:** A student with no actual result is excluded from accuracy metrics — delivered exactly as documented.
**Priority:** High

### TC-SA-TE-40-006 — Rule: bias of zero → no systematic error
**Type:** Edge
**Covers:** 5.1 → Rule: zero bias
**Preconditions:** Predictions are perfectly centered on actuals.
**Steps:**
1. Inspect the bias metric.
2. Verify a bias of zero indicates no systematic over/under prediction.
**Expected Result:** A bias metric of zero indicates no systematic over/under prediction — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-40-007 — Rule: below minimum exams → "insufficient data"
**Type:** Edge
**Covers:** 5.1 → Rule: insufficient data
**Preconditions:** A cohort has fewer than the minimum completed exams.
**Steps:**
1. Review the cohort accuracy.
2. Verify it shows "insufficient data".
**Expected Result:** A cohort with fewer than the minimum completed exams shows "insufficient data" — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-40-008 — Rule: never estimated for missing actuals
**Type:** Negative
**Covers:** 5.1 → Rule: no estimates
**Preconditions:** Some students have missing actuals.
**Steps:**
1. Review the accuracy metrics.
2. Verify no accuracy metric is estimated for missing actuals.
**Expected Result:** An accuracy metric is never estimated for missing actuals — delivered exactly as documented.
**Priority:** High

### TC-SA-TE-40-009 — Audit logging of prediction accuracy analytics
**Type:** Positive
**Covers:** 5.1 → Audit logging
**Preconditions:** A Super Admin account is active.
**Steps:**
1. Change a prediction accuracy analytics setting.
2. Verify the change is recorded in the audit log with actor, timestamp, and before/after values.
**Expected Result:** Audit logging of prediction accuracy analytics — delivered exactly as documented.
**Priority:** Medium

## 5.2 Plan Adherence Analytics

### TC-SA-TE-41-001 — Adherence rate (on-time completion %)
**Type:** Positive
**Covers:** 5.2 → Adherence rate
**Preconditions:** A Super Admin account is active; students have study plans.
**Steps:**
1. Open Analytics → Plan Adherence.
2. Review the adherence rate (on-time task completion percentage).
**Expected Result:** Adherence rate — delivered exactly as documented.
**Priority:** High

### TC-SA-TE-41-002 — Slippage analysis (average days)
**Type:** Positive
**Covers:** 5.2 → Slippage analysis
**Preconditions:** Students have study plans with completed tasks.
**Steps:**
1. Inspect the slippage analysis (average days tasks slip).
2. Verify the slippage is computed correctly.
**Expected Result:** Slippage analysis — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-41-003 — Adherence by exam type, cohort, horizon
**Type:** Positive
**Covers:** 5.2 → Adherence by slice
**Preconditions:** Students have study plans.
**Steps:**
1. Slice adherence by exam type, cohort, and plan horizon.
2. Verify the sliced metrics are correct.
**Expected Result:** Adherence by exam type, cohort, and plan horizon — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-41-004 — At-risk and off-track identification
**Type:** Positive
**Covers:** 5.2 → At-risk/off-track
**Preconditions:** Students have study plans.
**Steps:**
1. Identify at-risk and off-track students.
2. Verify the identification matches the thresholds.
**Expected Result:** At-risk and off-track student identification — delivered exactly as documented.
**Priority:** High

### TC-SA-TE-41-005 — Rule: no completed tasks → 0% not excluded
**Type:** Edge
**Covers:** 5.2 → Rule: 0%
**Preconditions:** A plan has no completed tasks.
**Steps:**
1. Review the adherence rate.
2. Verify the plan shows 0% adherence (not excluded).
**Expected Result:** A plan with no completed tasks shows 0% adherence (not excluded) — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-41-006 — Rule: slippage over completed tasks only
**Type:** Edge
**Covers:** 5.2 → Rule: completed only
**Preconditions:** A plan has both completed and pending tasks.
**Steps:**
1. Inspect the slippage metric.
2. Verify it is computed only over completed tasks.
**Expected Result:** A slippage metric is computed only over completed tasks — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-41-007 — Rule: at-risk = at threshold not off-track
**Type:** Edge
**Covers:** 5.2 → Rule: at-risk
**Preconditions:** A student is at the at-risk threshold.
**Steps:**
1. Review the student's classification.
2. Verify the student is at-risk (not yet off-track).
**Expected Result:** An at-risk student is one at the at-risk threshold (not yet off-track) — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-41-008 — Rule: no active plans → "no data"
**Type:** Edge
**Covers:** 5.2 → Rule: no data
**Preconditions:** A cohort has no active plans.
**Steps:**
1. Review the cohort adherence.
2. Verify it shows "no data".
**Expected Result:** A cohort adherence with no active plans shows "no data" — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-41-009 — Audit logging of plan adherence analytics
**Type:** Positive
**Covers:** 5.2 → Audit logging
**Preconditions:** A Super Admin account is active.
**Steps:**
1. Change a plan adherence analytics setting.
2. Verify the change is recorded in the audit log with actor, timestamp, and before/after values.
**Expected Result:** Audit logging of plan adherence analytics — delivered exactly as documented.
**Priority:** Medium

## 5.3 Mock Test Trend Analytics

### TC-SA-TE-42-001 — Score trend (per student across mocks)
**Type:** Positive
**Covers:** 5.3 → Score trend
**Preconditions:** A Super Admin account is active; students have mock test results.
**Steps:**
1. Open Analytics → Mock Test Trends.
2. Review the score trend per student across mocks.
**Expected Result:** Score trend — delivered exactly as documented.
**Priority:** High

### TC-SA-TE-42-002 — Improvement rate (score delta)
**Type:** Positive
**Covers:** 5.3 → Improvement rate
**Preconditions:** Students have multiple mock results.
**Steps:**
1. Inspect the improvement rate (score delta between mocks).
2. Verify the improvement rate is computed correctly.
**Expected Result:** Improvement rate — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-42-003 — Trend by exam type and cohort
**Type:** Positive
**Covers:** 5.3 → Trend by slice
**Preconditions:** Students have mock results.
**Steps:**
1. Slice the trend by exam type and cohort.
2. Verify the sliced trends are correct.
**Expected Result:** Trend by exam type and cohort — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-42-004 — Plateau detection (no improvement over N)
**Type:** Positive
**Covers:** 5.3 → Plateau detection
**Preconditions:** Students have multiple mock results.
**Steps:**
1. Identify plateauing students.
2. Verify plateau detection matches the configured criteria.
**Expected Result:** Plateau detection — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-42-005 — Rule: fewer than two mocks → no trend
**Type:** Edge
**Covers:** 5.3 → Rule: insufficient
**Preconditions:** A student has fewer than two mocks.
**Steps:**
1. Review the student's trend.
2. Verify no trend is shown (insufficient data).
**Expected Result:** A student with fewer than two mocks shows no trend (insufficient data) — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-42-006 — Rule: plateau after configured N mocks
**Type:** Edge
**Covers:** 5.3 → Rule: N mocks
**Preconditions:** The plateau threshold N is configured.
**Steps:**
1. Review a student with no improvement.
2. Verify a plateau is detected only after the configured number of mocks (N).
**Expected Result:** A plateau is detected only after the configured number of mocks (N) — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-42-007 — Rule: delta on comparable mocks only
**Type:** Edge
**Covers:** 5.3 → Rule: comparable
**Preconditions:** A student has mocks of different exam types.
**Steps:**
1. Inspect the score delta.
2. Verify it is computed on comparable mocks (same exam type).
**Expected Result:** A score delta is computed on comparable mocks (same exam type) — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-42-008 — Rule: no mock data → "no data"
**Type:** Edge
**Covers:** 5.3 → Rule: no data
**Preconditions:** A cohort has no mock data.
**Steps:**
1. Review the cohort trend.
2. Verify it shows "no data".
**Expected Result:** A cohort trend with no mock data shows "no data" — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-42-009 — Audit logging of mock test trend analytics
**Type:** Positive
**Covers:** 5.3 → Audit logging
**Preconditions:** A Super Admin account is active.
**Steps:**
1. Change a mock test trend analytics setting.
2. Verify the change is recorded in the audit log with actor, timestamp, and before/after values.
**Expected Result:** Audit logging of mock test trend analytics — delivered exactly as documented.
**Priority:** Medium

## 5.4 Gap Closure Analytics

### TC-SA-TE-43-001 — Gap identification (weak topics)
**Type:** Positive
**Covers:** 5.4 → Gap identification
**Preconditions:** A Super Admin account is active; students have mock and practice data.
**Steps:**
1. Open Analytics → Gap Closure.
2. Review the gap identification (weak topics from mocks and practice).
**Expected Result:** Gap identification — delivered exactly as documented.
**Priority:** High

### TC-SA-TE-43-002 — Gap closure rate
**Type:** Positive
**Covers:** 5.4 → Gap closure rate
**Preconditions:** Gaps have been identified.
**Steps:**
1. Inspect the gap closure rate (weak topics improved to target).
2. Verify the closure rate is computed correctly.
**Expected Result:** Gap closure rate — delivered exactly as documented.
**Priority:** High

### TC-SA-TE-43-003 — Gap persistence
**Type:** Positive
**Covers:** 5.4 → Gap persistence
**Preconditions:** Gaps have been identified.
**Steps:**
1. Check the gap persistence (topics remaining weak over time).
2. Verify persistence is tracked correctly.
**Expected Result:** Gap persistence — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-43-004 — Gap closure by exam type and cohort
**Type:** Positive
**Covers:** 5.4 → Closure by slice
**Preconditions:** Gaps have been identified.
**Steps:**
1. Slice the closure by exam type and cohort.
2. Verify the sliced metrics are correct.
**Expected Result:** Gap closure by exam type and cohort — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-43-005 — Rule: closed only at target threshold
**Type:** Edge
**Covers:** 5.4 → Rule: threshold
**Preconditions:** A topic's accuracy is below the target threshold.
**Steps:**
1. Review the gap status.
2. Verify the gap is closed only when the topic accuracy reaches the target threshold.
**Expected Result:** A gap is closed only when the topic accuracy reaches the target threshold — delivered exactly as documented.
**Priority:** High

### TC-SA-TE-43-006 — Rule: persistent gap flagged for intervention
**Type:** Edge
**Covers:** 5.4 → Rule: persistent
**Preconditions:** A topic remains weak over the configured period.
**Steps:**
1. Review the gap status.
2. Verify the persistent gap is flagged for intervention.
**Expected Result:** A persistent gap (weak over the configured period) is flagged for intervention — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-43-007 — Rule: no gaps → "no gaps"
**Type:** Edge
**Covers:** 5.4 → Rule: no gaps
**Preconditions:** No gaps have been identified.
**Steps:**
1. Review the gap closure rate.
2. Verify it shows "no gaps".
**Expected Result:** A gap closure rate with no identified gaps shows "no gaps" — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-43-008 — Rule: insufficient attempts → not counted
**Type:** Edge
**Covers:** 5.4 → Rule: insufficient attempts
**Preconditions:** A topic has insufficient attempts.
**Steps:**
1. Review the gap status.
2. Verify the topic is not counted as closed or persistent.
**Expected Result:** A topic with insufficient attempts is not counted as closed or persistent — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-43-009 — Audit logging of gap closure analytics
**Type:** Positive
**Covers:** 5.4 → Audit logging
**Preconditions:** A Super Admin account is active.
**Steps:**
1. Change a gap closure analytics setting.
2. Verify the change is recorded in the audit log with actor, timestamp, and before/after values.
**Expected Result:** Audit logging of gap closure analytics — delivered exactly as documented.
**Priority:** Medium

## 5.5 Cohort & Institution-Level Reporting

### TC-SA-TE-44-001 — Cohort report (aggregated metrics)
**Type:** Positive
**Covers:** 5.5 → Cohort report
**Preconditions:** A Super Admin account is active; cohort data exists.
**Steps:**
1. Open Analytics → Reports.
2. Generate a cohort report.
3. Verify the aggregated prediction, adherence, mock, and gap metrics are correct.
**Expected Result:** Cohort report — delivered exactly as documented.
**Priority:** High

### TC-SA-TE-44-002 — Institution report (comparable cohorts)
**Type:** Positive
**Covers:** 5.5 → Institution report
**Preconditions:** Multiple institutions have cohort data.
**Steps:**
1. Generate an institution report with comparable cohorts.
2. Verify the per-institution comparison is correct.
**Expected Result:** Institution report — delivered exactly as documented.
**Priority:** High

### TC-SA-TE-44-003 — Report export (PDF, CSV)
**Type:** Positive
**Covers:** 5.5 → Report export
**Preconditions:** A report has been generated.
**Steps:**
1. Export the report as PDF and CSV.
2. Verify the exports contain the report data.
**Expected Result:** Report export — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-44-004 — Report scheduling (periodic delivery)
**Type:** Positive
**Covers:** 5.5 → Report scheduling
**Preconditions:** A report has been generated.
**Steps:**
1. Schedule periodic report delivery.
2. Verify the report is delivered on schedule.
**Expected Result:** Report scheduling — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-44-005 — Rule: no students → "no data" not empty chart
**Type:** Edge
**Covers:** 5.5 → Rule: no data
**Preconditions:** A cohort has no students.
**Steps:**
1. Generate the cohort report.
2. Verify it shows "no data" (not an empty chart).
**Expected Result:** A cohort report with no students shows "no data" (not an empty chart) — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-44-006 — Rule: only comparable exam types compared
**Type:** Edge
**Covers:** 5.5 → Rule: comparable
**Preconditions:** Institutions have cohorts with different exam types.
**Steps:**
1. Generate the institution report.
2. Verify only cohorts with comparable exam types are compared.
**Expected Result:** An institution report compares only cohorts with comparable exam types — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-44-007 — Rule: no data → header-only export
**Type:** Edge
**Covers:** 5.5 → Rule: header-only
**Preconditions:** A report has no data.
**Steps:**
1. Export the report.
2. Verify it exports a header-only file (not an error).
**Expected Result:** A report export with no data exports a header-only file (not an error) — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-44-008 — Rule: scheduled delivery respects email prefs
**Type:** Edge
**Covers:** 5.5 → Rule: email prefs
**Preconditions:** A recipient has email notifications disabled.
**Steps:**
1. Trigger a scheduled report delivery.
2. Verify the recipient is not emailed.
**Expected Result:** A scheduled report delivery respects the recipient's email preferences — delivered exactly as documented.
**Priority:** Medium

### TC-SA-TE-44-009 — Audit logging of cohort & institution-level reporting
**Type:** Positive
**Covers:** 5.5 → Audit logging
**Preconditions:** A Super Admin account is active.
**Steps:**
1. Change a cohort & institution-level reporting setting.
2. Verify the change is recorded in the audit log with actor, timestamp, and before/after values.
**Expected Result:** Audit logging of cohort & institution-level reporting — delivered exactly as documented.
**Priority:** Medium