# 6. Content Effectiveness Tracking — Test Cases

User Type: **Super Administrator**
Source: *Mi Digital Academy - Education CRM Features Document*
Spec: content_effectiveness_tracking.md — every feature, sub-feature, and rule covered

---

## Test Execution Policy
- Zero tolerance: any deviation from documented behavior = FAILED = bug
- Every bug is immediately logged/reported (Bug ID, feature, sub-feature,
  expected vs actual, severity) and fixed 100% before the group passes
- Feature group passes only at 100% test pass rate

## Coverage Matrix
| Feature | Sub-feature / Rule | Test IDs |
|---------|--------------------|----------|
| 6.1 Video Completion Rates | Completion rate: the proportion of starters who complete a video | TC-SA-07-06-001 |
| 6.1 Video Completion Rates | Per-video metric: the rate measured for each video | TC-SA-07-06-002 |
| 6.1 Video Completion Rates | Trend: the completion rate over time (improving or declining) | TC-SA-07-06-003 |
| 6.1 Video Completion Rates | Comparison: the rates across videos (the strong vs. the weak) | TC-SA-07-06-004 |
| 6.1 Video Completion Rates | Threshold flag: a video below a completion threshold flagged for review | TC-SA-07-06-005 |
| 6.1 Video Completion Rates | Context: the rate in the context of the video's length and difficulty | TC-SA-07-06-006 |
| 6.1 Video Completion Rates | Audit/logging of the metric reviews (where applicable) | TC-SA-07-06-007 |
| 6.1 Video Completion Rates | Rule: the completion rate is the proportion of starters who complete; it measures engagement, not just reach | TC-SA-07-06-008 |
| 6.1 Video Completion Rates | Rule: the rate is per video and over time (the trend); a single snapshot is not the whole picture | TC-SA-07-06-003 |
| 6.1 Video Completion Rates | Rule: the rate is read in context (length, difficulty); a raw low rate is not automatically a failure | TC-SA-07-06-006 |
| 6.1 Video Completion Rates | Rule: a threshold flag marks a video for review; the flag is a signal, not an automatic action | TC-SA-07-06-005 |
| 6.1 Video Completion Rates | Rule: a content fix is verified by the rate's response; a persistent low rate escalates | TC-SA-07-06-009 |
| 6.1 Video Completion Rates | Rule: metric reviews are logged where applicable | TC-SA-07-06-007 |
| 6.2 Drop-off Points in Videos | Drop-off detection: the points in a video where learners stop watching | TC-SA-07-06-010 |
| 6.2 Drop-off Points in Videos | Drop-off curve: the retention over the video's timeline (where it falls) | TC-SA-07-06-011 |
| 6.2 Drop-off Points in Videos | Peak drop-off: the specific moment(s) of highest abandonment | TC-SA-07-06-012 |
| 6.2 Drop-off Points in Videos | Per-video analysis: the drop-off points for each video | TC-SA-07-06-013 |
| 6.2 Drop-off Points in Videos | Correlation: the drop-off points correlated with the content (a section, a topic) | TC-SA-07-06-014 |
| 6.2 Drop-off Points in Videos | Targeted fix: the content at the drop-off point addressed specifically | TC-SA-07-06-015 |
| 6.2 Drop-off Points in Videos | Audit/logging of the analysis (where applicable) | TC-SA-07-06-016 |
| 6.2 Drop-off Points in Videos | Rule: the drop-off is measured over the video's timeline; the curve shows where retention falls | TC-SA-07-06-011 |
| 6.2 Drop-off Points in Videos | Rule: the peak drop-off is the specific moment of highest abandonment; it is located, not estimated | TC-SA-07-06-012 |
| 6.2 Drop-off Points in Videos | Rule: the drop-off is correlated with the content (the section/topic at that point); the cause is identified, not guessed | TC-SA-07-06-014 |
| 6.2 Drop-off Points in Videos | Rule: a technical cause and a content cause are distinguished; the fix is targeted to the cause | TC-SA-07-06-017 |
| 6.2 Drop-off Points in Videos | Rule: the fix is verified by the drop-off curve's response; a persistent peak escalates | TC-SA-07-06-018 |
| 6.2 Drop-off Points in Videos | Rule: the analysis is logged where applicable | TC-SA-07-06-016 |
| 6.3 Student Feedback Scores on Content | Feedback capture: the learners' ratings and comments on content items | TC-SA-07-06-019 |
| 6.3 Student Feedback Scores on Content | Per-content score: the feedback score for each content item | TC-SA-07-06-020 |
| 6.3 Student Feedback Scores on Content | Rating scale: the rating the learners use (e.g., a star scale) | TC-SA-07-06-021 |
| 6.3 Student Feedback Scores on Content | Comments: the qualitative feedback (the learners' words) | TC-SA-07-06-022 |
| 6.3 Student Feedback Scores on Content | Trend: the score over time (improving or declining) | TC-SA-07-06-023 |
| 6.3 Student Feedback Scores on Content | Low-score flag: a content item with a low score flagged for review | TC-SA-07-06-024 |
| 6.3 Student Feedback Scores on Content | Correlation: the feedback correlated with the behavioral metrics (completion, drop-off) | TC-SA-07-06-025 |
| 6.3 Student Feedback Scores on Content | Audit/logging of the feedback reviews (where applicable) | TC-SA-07-06-026 |
| 6.3 Student Feedback Scores on Content | Rule: the feedback is the learners' ratings and comments; it is the audience's perspective, complementing the behavioral metrics | TC-SA-07-06-019 |
| 6.3 Student Feedback Scores on Content | Rule: the score is per content item and over time (the trend); a single snapshot is not the whole picture | TC-SA-07-06-023 |
| 6.3 Student Feedback Scores on Content | Rule: the comments are read for specificity; a low score without comments is investigated (why no feedback) | TC-SA-07-06-027 |
| 6.3 Student Feedback Scores on Content | Rule: a low score is correlated with the behavioral metrics; the correlation distinguishes a real problem from a perception issue | TC-SA-07-06-028 |
| 6.3 Student Feedback Scores on Content | Rule: a low-score flag marks the content for review; the flag is a signal, not an automatic action | TC-SA-07-06-024 |
| 6.3 Student Feedback Scores on Content | Rule: feedback reviews are logged where applicable | TC-SA-07-06-026 |

## 6.1 Video Completion Rates

### TC-SA-07-06-001 — Completion rate: the proportion of starters who complete a video
**Type:** Positive
**Covers:** 6.1 → Completion rate: the proportion of starters who complete a video
**Preconditions:** Learners have started and completed a video.
**Steps:**
1. Open the content effectiveness view.
2. Review the video completion rate: the proportion of learners who complete the video (watch it through) versus those who start it.
**Expected Result:** The completion rate is shown — the proportion of starters who complete the video.
**Priority:** Critical

### TC-SA-07-06-002 — Per-video metric: the rate measured for each video
**Type:** Positive
**Covers:** 6.1 → Per-video metric: the rate measured for each video
**Preconditions:** Multiple videos have learner activity.
**Steps:**
1. Open the content effectiveness view.
2. Verify the completion rate is measured for each video (per-video metric).
**Expected Result:** The completion rate is measured per video — each video has its own rate.
**Priority:** High

### TC-SA-07-06-003 — Trend: the completion rate over time (improving or declining); a single snapshot is not the whole picture
**Type:** Positive
**Covers:** 6.1 → Trend: the completion rate over time (improving or declining); Rule: the rate is per video and over time (the trend); a single snapshot is not the whole picture
**Preconditions:** A video has completion data over a period.
**Steps:**
1. Review the trend: the completion rate over time.
2. Verify the trend shows whether the rate is improving or declining.
3. Verify a decline suggests a recent issue; a stable low rate suggests an inherent content problem.
**Expected Result:** The trend shows the completion rate over time — a single snapshot is not the whole picture.
**Priority:** High

### TC-SA-07-06-004 — Comparison: the rates across videos (the strong vs. the weak)
**Type:** Positive
**Covers:** 6.1 → Comparison: the rates across videos (the strong vs. the weak)
**Preconditions:** Multiple videos have completion data.
**Steps:**
1. Compare the rates across the videos.
2. Verify the strong performers (high completion) versus the weak ones are clear.
**Expected Result:** The rates are compared across the videos — the strong vs. the weak are clear.
**Priority:** High

### TC-SA-07-06-005 — Threshold flag: a video below a completion threshold flagged for review; the flag is a signal, not an automatic action
**Type:** Negative
**Covers:** 6.1 → Threshold flag: a video below a completion threshold flagged for review; Rule: a threshold flag marks a video for review; the flag is a signal, not an automatic action
**Preconditions:** A video's completion rate is below the threshold.
**Steps:**
1. Verify the video below the completion threshold is flagged for review.
2. Verify the flag is a signal (routed for review), not an automatic action.
**Expected Result:** The video below the completion threshold is flagged for review — the flag is a signal, not an automatic action.
**Priority:** High

### TC-SA-07-06-006 — Context: the rate in the context of the video's length and difficulty; a raw low rate is not automatically a failure
**Type:** Positive
**Covers:** 6.1 → Context: the rate in the context of the video's length and difficulty; Rule: the rate is read in context (length, difficulty); a raw low rate is not automatically a failure
**Preconditions:** A long, advanced video has a lower completion rate than a short, beginner one.
**Steps:**
1. Put the rate in context: the video's length and difficulty.
2. Verify the rate is read in context (a long, advanced video has a naturally lower rate than a short, beginner one).
3. Verify a raw low rate is not automatically a failure.
**Expected Result:** The rate is read in context (length, difficulty) — a raw low rate is not automatically a failure.
**Priority:** Medium

### TC-SA-07-06-007 — Metric reviews are logged where applicable (the video, the review, the timestamp)
**Type:** Positive
**Covers:** 6.1 → Audit/logging of the metric reviews (where applicable); Rule: metric reviews are logged where applicable
**Preconditions:** Super Admin has reviewed video completion rates.
**Steps:**
1. Open the log and filter by "metric review".
2. Verify entries show the video, the review, and the timestamp (where applicable).
**Expected Result:** The metric reviews are logged where applicable — the video, the review, and the timestamp.
**Priority:** Medium

### TC-SA-07-06-008 — The completion rate is the proportion of starters who complete; it measures engagement, not just reach
**Type:** Edge
**Covers:** 6.1 → Rule: the completion rate is the proportion of starters who complete; it measures engagement, not just reach
**Preconditions:** A video has many starts but few completions.
**Steps:**
1. Verify the completion rate is computed as the proportion of starters who complete (not the proportion of all enrolled learners).
2. Verify the rate measures engagement, not just reach.
**Expected Result:** The completion rate is the proportion of starters who complete — it measures engagement, not just reach.
**Priority:** High

### TC-SA-07-06-009 — A content fix is verified by the rate's response; a persistent low rate escalates
**Type:** Edge
**Covers:** 6.1 → Rule: a content fix is verified by the rate's response; a persistent low rate escalates
**Preconditions:** A weak video is routed for a content fix (a re-cut to a shorter length); the fix is applied.
**Steps:**
1. After the content fix, review the rate again.
2. Verify an improvement confirms the fix worked.
3. Verify a persistent low rate escalates the review.
**Expected Result:** The content fix is verified by the rate's response — a persistent low rate escalates.
**Priority:** High

## 6.2 Drop-off Points in Videos

### TC-SA-07-06-010 — Drop-off detection: the points in a video where learners stop watching
**Type:** Positive
**Covers:** 6.2 → Drop-off detection: the points in a video where learners stop watching
**Preconditions:** A video has learner viewing data with abandonment.
**Steps:**
1. Open the drop-off analysis for the video.
2. Verify the points in the video where learners stop watching are detected.
**Expected Result:** The drop-off points (where learners stop watching) are detected.
**Priority:** Critical

### TC-SA-07-06-011 — Drop-off curve: the retention over the video's timeline (where it falls); the curve shows where retention falls
**Type:** Positive
**Covers:** 6.2 → Drop-off curve: the retention over the video's timeline (where it falls); Rule: the drop-off is measured over the video's timeline; the curve shows where retention falls
**Preconditions:** A video has learner viewing data.
**Steps:**
1. Open the drop-off analysis: the retention curve over the video's timeline.
2. Verify the curve shows where retention falls.
**Expected Result:** The drop-off curve shows the retention over the video's timeline — where it falls.
**Priority:** High

### TC-SA-07-06-012 — Peak drop-off: the specific moment(s) of highest abandonment; it is located, not estimated
**Type:** Positive
**Covers:** 6.2 → Peak drop-off: the specific moment(s) of highest abandonment; Rule: the peak drop-off is the specific moment of highest abandonment; it is located, not estimated
**Preconditions:** A video has a sharp retention fall at a specific moment (e.g., at the 8-minute mark).
**Steps:**
1. Identify the peak drop-off: the specific moment where the retention falls sharply.
2. Verify the peak is located (a specific moment), not estimated.
**Expected Result:** The peak drop-off is the specific moment of highest abandonment — it is located, not estimated.
**Priority:** High

### TC-SA-07-06-013 — Per-video analysis: the drop-off points for each video
**Type:** Positive
**Covers:** 6.2 → Per-video analysis: the drop-off points for each video
**Preconditions:** Multiple videos have learner viewing data.
**Steps:**
1. Review the drop-off points across the videos.
2. Verify the drop-off points are analyzed for each video (per-video analysis).
**Expected Result:** The drop-off points are analyzed per video — each video has its own drop-off analysis.
**Priority:** Medium

### TC-SA-07-06-014 — Correlation: the drop-off points correlated with the content (a section, a topic); the cause is identified, not guessed
**Type:** Positive
**Covers:** 6.2 → Correlation: the drop-off points correlated with the content (a section, a topic); Rule: the drop-off is correlated with the content (the section/topic at that point); the cause is identified, not guessed
**Preconditions:** A video has a peak drop-off at a specific moment.
**Steps:**
1. Correlate the drop-off with the content: what is happening at that point (a complex explanation, a long demonstration, a transition).
2. Verify the cause is identified (not guessed).
**Expected Result:** The drop-off is correlated with the content — the cause is identified, not guessed.
**Priority:** High

### TC-SA-07-06-015 — Targeted fix: the content at the drop-off point addressed specifically
**Type:** Positive
**Covers:** 6.2 → Targeted fix: the content at the drop-off point addressed specifically
**Preconditions:** A drop-off point is identified with a content cause (a confusing section).
**Steps:**
1. Route a content fix for the drop-off point (a clearer explanation, a visual aid, a split).
2. Verify the content at the drop-off point is addressed specifically (targeted, not guessed).
**Expected Result:** The content at the drop-off point is addressed specifically — the fix is targeted.
**Priority:** High

### TC-SA-07-06-016 — The analysis is logged where applicable (the video, the drop-off point, the action, the timestamp)
**Type:** Positive
**Covers:** 6.2 → Audit/logging of the analysis (where applicable); Rule: the analysis is logged where applicable
**Preconditions:** Super Admin has performed drop-off analysis.
**Steps:**
1. Open the log and filter by "drop-off analysis".
2. Verify entries show the video, the drop-off point, the action, and the timestamp (where applicable).
**Expected Result:** The analysis is logged where applicable — the video, the drop-off point, the action, and the timestamp.
**Priority:** Medium

### TC-SA-07-06-017 — A technical cause and a content cause are distinguished; the fix is targeted to the cause
**Type:** Negative
**Covers:** 6.2 → Rule: a technical cause and a content cause are distinguished; the fix is targeted to the cause
**Preconditions:** A drop-off point has a technical cause (a glitch, a bad segment at that point).
**Steps:**
1. Identify the technical cause (a glitch, a bad segment at that point).
2. Distinguish it from a content cause.
3. Route a fix targeted to the cause (a re-record of that segment).
**Expected Result:** The technical cause and a content cause are distinguished — the fix is targeted to the cause.
**Priority:** High

### TC-SA-07-06-018 — The fix is verified by the drop-off curve's response; a persistent peak escalates
**Type:** Edge
**Covers:** 6.2 → Rule: the fix is verified by the drop-off curve's response; a persistent peak escalates
**Preconditions:** A targeted fix is applied at a drop-off point.
**Steps:**
1. After the fix, review the drop-off curve again.
2. Verify the peak at that point reduced confirms the targeted fix worked.
3. Verify a persistent peak escalates.
**Expected Result:** The fix is verified by the drop-off curve's response — a persistent peak escalates.
**Priority:** High

## 6.3 Student Feedback Scores on Content

### TC-SA-07-06-019 — Feedback capture: the learners' ratings and comments on content items; it is the audience's perspective, complementing the behavioral metrics
**Type:** Positive
**Covers:** 6.3 → Feedback capture: the learners' ratings and comments on content items; Rule: the feedback is the learners' ratings and comments; it is the audience's perspective, complementing the behavioral metrics
**Preconditions:** Learners have rated and commented on content items.
**Steps:**
1. Review the student feedback scores on the content: the ratings and comments per content item.
2. Verify the feedback is the learners' ratings and comments (the audience's perspective).
3. Verify it complements the behavioral metrics (completion, drop-off).
**Expected Result:** The feedback is captured — the learners' ratings and comments on content items, complementing the behavioral metrics.
**Priority:** Critical

### TC-SA-07-06-020 — Per-content score: the feedback score for each content item
**Type:** Positive
**Covers:** 6.3 → Per-content score: the feedback score for each content item
**Preconditions:** Multiple content items have learner feedback.
**Steps:**
1. Review the feedback scores.
2. Verify the feedback score is shown for each content item (per-content score).
**Expected Result:** The feedback score is shown per content item.
**Priority:** High

### TC-SA-07-06-021 — Rating scale: the rating the learners use (e.g., a star scale)
**Type:** Positive
**Covers:** 6.3 → Rating scale: the rating the learners use (e.g., a star scale)
**Preconditions:** A learner is rating a content item.
**Steps:**
1. As the learner, rate the content item using the rating scale (e.g., a star scale).
2. Verify the rating is captured on the scale.
**Expected Result:** The rating scale (e.g., a star scale) is used — the learner's rating is captured.
**Priority:** Medium

### TC-SA-07-06-022 — Comments: the qualitative feedback (the learners' words)
**Type:** Positive
**Covers:** 6.3 → Comments: the qualitative feedback (the learners' words)
**Preconditions:** A learner has left a comment on a content item.
**Steps:**
1. Review the comments: the qualitative feedback (the learners' words).
2. Verify the comments are captured and readable (what is wrong: unclear, too fast, not relevant).
**Expected Result:** The comments (the qualitative feedback) are captured — the learners' words are available.
**Priority:** High

### TC-SA-07-06-023 — Trend: the score over time (improving or declining); a single snapshot is not the whole picture
**Type:** Positive
**Covers:** 6.3 → Trend: the score over time (improving or declining); Rule: the score is per content item and over time (the trend); a single snapshot is not the whole picture
**Preconditions:** A content item has feedback scores over a period.
**Steps:**
1. Review the trend: the score over time.
2. Verify the trend shows whether the score is improving or declining.
3. Verify a decline after a content change suggests the change hurt; an improvement confirms a fix.
**Expected Result:** The trend shows the score over time — a single snapshot is not the whole picture.
**Priority:** High

### TC-SA-07-06-024 — Low-score flag: a content item with a low score flagged for review; the flag is a signal, not an automatic action
**Type:** Negative
**Covers:** 6.3 → Low-score flag: a content item with a low score flagged for review; Rule: a low-score flag marks the content for review; the flag is a signal, not an automatic action
**Preconditions:** A content item has a low score (the learners rate it poorly).
**Steps:**
1. Verify the content item with a low score is flagged for review.
2. Verify the flag is a signal (routed for review), not an automatic action.
**Expected Result:** The content item with a low score is flagged for review — the flag is a signal, not an automatic action.
**Priority:** High

### TC-SA-07-06-025 — Correlation: the feedback correlated with the behavioral metrics (completion, drop-off)
**Type:** Positive
**Covers:** 6.3 → Correlation: the feedback correlated with the behavioral metrics (completion, drop-off)
**Preconditions:** A content item has a low score and a low completion rate.
**Steps:**
1. Correlate the feedback with the behavioral metrics.
2. Verify a low score with a low completion rate confirms a real problem.
3. Verify a low score with a high completion rate suggests a perception issue.
**Expected Result:** The feedback is correlated with the behavioral metrics — the correlation distinguishes a real problem from a perception issue.
**Priority:** High

### TC-SA-07-06-026 — Feedback reviews are logged where applicable (the content, the score, the action, the timestamp)
**Type:** Positive
**Covers:** 6.3 → Audit/logging of the feedback reviews (where applicable); Rule: feedback reviews are logged where applicable
**Preconditions:** Super Admin has reviewed feedback scores.
**Steps:**
1. Open the log and filter by "feedback review".
2. Verify entries show the content, the score, the action, and the timestamp (where applicable).
**Expected Result:** The feedback reviews are logged where applicable — the content, the score, the action, and the timestamp.
**Priority:** Medium

### TC-SA-07-06-027 — A low score without comments is investigated (why no feedback)
**Type:** Edge
**Covers:** 6.3 → Rule: the comments are read for specificity; a low score without comments is investigated (why no feedback)
**Preconditions:** A content item has a low score but no comments.
**Steps:**
1. Identify the low score without comments.
2. Investigate why no feedback (comments) was left.
3. Verify the investigation is recorded.
**Expected Result:** The low score without comments is investigated — why no feedback is determined.
**Priority:** Medium

### TC-SA-07-06-028 — The correlation distinguishes a real problem from a perception issue
**Type:** Edge
**Covers:** 6.3 → Rule: a low score is correlated with the behavioral metrics; the correlation distinguishes a real problem from a perception issue
**Preconditions:** Content A has a low score and a low completion rate; Content B has a low score and a high completion rate.
**Steps:**
1. Correlate Content A's low score with its low completion rate and verify it confirms a real problem.
2. Correlate Content B's low score with its high completion rate and verify it suggests a perception issue.
**Expected Result:** The correlation distinguishes a real problem (low score + low completion) from a perception issue (low score + high completion).
**Priority:** High