AI Skill Report Card
Engineering QA Test Plans
QA Engineer: Test Planning & Coverage Analysis
Quick Start14 / 15
Given a feature/change description, produce four artifacts in order:
- Test Plan (scope, strategy, environments, entry/exit criteria)
- Test Cases (numbered, structured, traceable to requirements)
- Coverage Matrix (requirement × test case mapping)
- Risk Analysis (probability × impact, with mitigation)
Example trigger: "We're shipping a password reset feature with email OTP, need QA sign-off." → Produce all four sections below, scoped to that feature.
Recommendation▾
Name is not gerund form ('engineering-qa-test-plans' vs something like 'planning-qa-tests') — minor deduction
Workflow14 / 15
Progress:
- [ ] Step 1: Clarify scope — what changed, what's untouched but adjacent (regression risk)
- [ ] Step 2: Identify test types needed (functional, regression, integration, edge/negative, performance, security if relevant)
- [ ] Step 3: Write Test Plan
- [ ] Step 4: Derive Test Cases from requirements + edge cases + past defects
- [ ] Step 5: Build Coverage Matrix linking requirements → test cases → automation status
- [ ] Step 6: Run Risk Analysis — flag untested/low-coverage high-impact areas
- [ ] Step 7: Recommend go/no-go or additional coverage needed
1. Test Plan structure
- Objective: what is being validated and why
- Scope: in-scope / out-of-scope features
- Test Strategy: manual vs automated split, test levels (unit/integration/e2e)
- Environments: staging, device/browser matrix, data setup
- Entry Criteria: build stable, feature deployed, test data ready
- Exit Criteria: % pass rate, no open critical/high defects, coverage threshold met
- Schedule: rough phase breakdown
2. Test Cases structure
Each test case includes:
ID | Title | Preconditions | Steps | Expected Result | Priority (P0-P3) | Type (Functional/Negative/Edge/Regression) | Automation Status (Manual/Automated/To-Automate)
Always include: happy path, negative/invalid input, boundary values, concurrency/race conditions where relevant, permission/auth checks, and regression of adjacent features.
3. Coverage Matrix structure
Requirement ID | Requirement Description | Test Case IDs | Coverage % | Automated? | Gap Notes
Flag any requirement with 0 linked test cases immediately.
4. Risk Analysis structure
Risk | Probability (H/M/L) | Impact (H/M/L) | Risk Score | Mitigation/Test Coverage | Owner
Prioritize risks where impact is High regardless of probability. Always call out areas with weak automation coverage as a standing risk.
Recommendation▾
Add a second example for a more complex or different-domain feature to show versatility beyond auth/OTP flows
Example
Input: "Password reset via email OTP. OTP valid 5 min, 3 attempts max, rate limited to 5 requests/hour per account."
Output:
Test Plan
- Objective: Validate OTP-based password reset flow works securely and reliably.
- Scope: OTP generation, delivery, validation, expiry, attempt limiting, rate limiting. Out of scope: SMS OTP (not built yet).
- Strategy: 70% automated (API-level), 30% manual (email delivery, UX).
- Environments: Staging with test SMTP sandbox; Chrome/Safari/mobile web.
- Entry: Feature deployed to staging, test email inbox configured.
- Exit: 100% P0/P1 cases pass, 0 open critical defects.
Test Cases (excerpt)
| ID | Title | Steps | Expected | Priority | Type |
|---|---|---|---|---|---|
| TC-01 | Valid OTP resets password | Request OTP → enter correct code within 5 min | Password reset succeeds | P0 | Functional |
| TC-02 | Expired OTP rejected | Wait 6 min → enter code | Error: "OTP expired" | P0 | Edge |
| TC-03 | 4th attempt blocked | Enter wrong code 3x → try again | Account locked, error shown | P0 | Negative |
| TC-04 | Rate limit enforced | Request OTP 6x in 1 hour | 6th request rejected | P1 | Edge |
| TC-05 | OTP not reusable | Use valid OTP, reuse same code again | Second use rejected | P0 | Negative |
Coverage Matrix
| Req ID | Description | Test Cases | Coverage | Automated | Gap |
|---|---|---|---|---|---|
| R1 | OTP expires in 5 min | TC-02 | 100% | Yes | — |
| R2 | Max 3 attempts | TC-03 | 100% | Yes | — |
| R3 | Rate limit 5/hr | TC-04 | 100% | No | Needs automation |
Risk Analysis
| Risk | Prob | Impact | Score | Mitigation |
|---|---|---|---|---|
| Email delivery delay causes false OTP-expired reports | M | H | High | Add delivery-time monitoring, manual test across providers |
| Rate limit bypass via multiple IPs | L | H | Medium | Add IP+account combined limiting, security test case |
Best Practices
- Trace every requirement to at least one test case — no orphan requirements.
- Prioritize P0 cases around auth, payment, and data integrity paths.
- Always include negative and boundary cases, not just happy path.
- Flag manual-only coverage on critical paths as a risk, not just a note.
- Reuse/extend regression suite rather than writing cases from scratch each cycle.
- State exit criteria numerically (pass %, defect severity threshold) — avoid vague "looks good."
Common Pitfalls
- Don't write test cases without linking them to a requirement or coverage matrix row.
- Don't treat "high automation %" as automatically low risk — check what's automated, not just how much.
- Don't skip risk analysis on features that "seem simple" — simple features often have the least test coverage.
- Don't give a go/no-go recommendation without stating open defect severity and count.
- Don't omit regression scope — new features often break adjacent, seemingly unrelated flows.