AI Skill Report Card

Analyzing Sentry Logs for Playwright

A-85·Mar 25, 2026·Source: Web
15 / 15
REPRODUCTION BLUEPRINT
Environment: https://app.example.com/dashboard | Chrome 120 | 1920x1080
Initial State: Authenticated user (ID: user_123), dark_mode=true
Execution Steps:
1. Click "Export Data" button
2. Select "CSV Format" from dropdown
3. Submit form triggers POST /api/export
Assertion Point: ReferenceError: exportManager is not defined
Mocking Requirements: Mock POST /api/export to return 500 Internal Server Error
Recommendation
Compress the workflow section - the progress checklist and detailed steps are somewhat redundant
15 / 15

Progress:

  • Extract environment context from Tags
  • Identify initial application state from Context
  • Map breadcrumbs to chronological actions
  • Locate the exact error point
  • Flag required API mocks from failed requests
  • Format as structured blueprint

Step-by-Step Process

  1. Parse Environment Data

    • Extract URL, browser, OS from Tags section
    • Note viewport size if available
    • Capture release version
  2. Analyze Application State

    • User authentication status from Context
    • Feature flags or configuration from Tags
    • LocalStorage/Cookie dependencies
  3. Convert Breadcrumbs to Actions

    • Focus on: ui.click, ui.input, navigation, http
    • Map UI events to user actions
    • Note failed HTTP requests (4xx/5xx status)
  4. Identify Assertion Point

    • Extract exact exception name and message
    • Note line numbers from stack trace
    • Determine observable failure state
  5. Document Mock Requirements

    • List failed API endpoints
    • Include expected error responses
    • Note timing dependencies
Recommendation
Add an edge case example showing how to handle logs with no breadcrumbs or minimal context
18 / 20

Example 1: Input: Sentry log with breadcrumbs showing navigation to /checkout, form submission, and TypeError on payment processing Output:

REPRODUCTION BLUEPRINT
Environment: https://shop.com/checkout | Firefox 118 | Mobile 375x667
Initial State: Cart contains 2 items, guest checkout
Execution Steps:
1. Navigate to checkout page
2. Fill shipping form fields
3. Click "Continue to Payment"
4. Enter credit card details
5. Submit payment form
Assertion Point: TypeError: Cannot read property 'validate' of null
Mocking Requirements: Mock POST /api/payment/validate to return 422 validation error

Example 2: Input: Insufficient breadcrumb data, only shows final error without user actions Output:

REPRODUCTION BLUEPRINT
Environment: https://admin.app.com/users | Chrome 119 | 1440x900
Initial State: Admin user authenticated
Execution Steps:
INSUFFICIENT DATA FOR: User interaction sequence leading to error
Assertion Point: ReferenceError: userManager is not defined at line 247
Mocking Requirements: None identified
Recommendation
Include a template section with a blank blueprint format that users can copy and fill
  • Extract only explicit data - Never invent selectors or API endpoints not shown in logs
  • Preserve exact error messages - Copy exception text verbatim for assertion matching
  • Flag data gaps - Use "INSUFFICIENT DATA FOR [X]" when logs incomplete
  • Map HTTP failures - Any 4xx/5xx response needs mocking strategy
  • Include timing context - Note if errors occur after delays or async operations
  • Document prerequisites - Authentication, feature flags, or setup requirements
  • Don't guess CSS selectors - Use descriptive text like "Login button" instead of "#login-btn"
  • Don't assume user paths - Multiple scenarios may lead to same error
  • Don't ignore environment differences - Mobile vs desktop can affect reproduction
  • Don't skip failed requests - Missing mocks cause different test failures
  • Don't paraphrase errors - Exact exception text needed for reliable assertions
  • Don't merge unrelated events - Separate distinct user workflows into scenarios
0
Grade A-AI Skill Framework
Scorecard
Criteria Breakdown
Quick Start
15/15
Workflow
15/15
Examples
18/20
Completeness
19/20
Format
15/15
Conciseness
13/15