Back to marketplace

TestRail Case and CI Result Sync

Sync automated tests to TestRail cases and scoped runs with stable mappings, retry handling, and verified result counts.

Development / Testingtestrailtest-management
by AgentVoltv1.0.0
Free to sign up · every skill included with AgentVolt Pro

Before & after

Same prompt, same agent — the only difference is the skill.

Prompt“Sync our test results to TestRail for the payment flow tests”
Beforegeneric AI agent
I can help you sync your test results to TestRail. To do that, I'll need to know a bit about your test setup. What testing framework are you using - Jest, Cypress, Selenium, something else? Do you have a TestRail project and test suite already set up? What's the project ID? Do you want to sync all tests or just the payment flow tests? Are you running these tests in CI or locally? Once I understand your setup and have your TestRail API credentials (or connection details), I can help you configure the sync, though you may need to run the integration manually the first time to make sure it's working correctly.
Afterwith TestRail Case and CI Result Sync
TESTRAIL SYNC - PAYMENT FLOW TESTS

SYNC CONFIGURATION COMPLETED

TEST METADATA:
  Framework: Jest with Cypress for E2E
  Test suite: payment-flow-tests
  Total test cases: 28
  Payment flow tests: 24 | Other: 4
  Execution environment: GitHub Actions CI
  Last run: 2026-07-31 14:23 UTC

TESTRAIL PROJECT LINKED:
  Project: AgentVolt (id: 12345)
  Test run name: "Payment Flow - Build #847"
  Run status: IN PROGRESS

TEST RESULTS SYNCED:

  PASSING (20 tests):
    ✓ Checkout form validation
    ✓ Credit card field accepts valid input
    ✓ Expiry validation (MM/YY format)
    ✓ CVV required for international cards
    ✓ Billing address auto-fills from shipping
    ✓ Discount code applied correctly (5 tests total)
    ✓ Payment confirmation email sent
    ✓ Receipt page displays order number
    ✓ Order appears in dashboard within 2 seconds
    ✓ Refund flow completes (partial & full)
    ✓ Retry on failed payment (auto-retry 3x)
    ... [20 tests total]

  FAILING (2 tests):
    ✗ "3D Secure popup closes on success"
      Issue: Popup doesn't auto-dismiss (requires user click)
      Status in TestRail: BLOCKED
      Assigned to: @marcus (payment lead)
      Priority: HIGH
      Error message: timeout waiting for element
      Screenshot attached: test-failure-3d-secure.png
      Retest recommendation: Run with 5s longer wait

    ✗ "Apply gift card during checkout"
      Issue: "Invalid gift card format" error on valid card
      Status in TestRail: FAILED
      Assigned to: @priya (backend)
      Priority: CRITICAL
      Root cause: Regex validation on backend too strict
      Fix PR: draft-gift-card-regex-fix

  SKIPPED (2 tests):
    ~ "Apple Pay flow" (requires physical Mac)
    ~ "Klarna installments" (account not provisioned)

TESTRAIL RUN METRICS:
  Pass rate: 83% (20/24)
  Execution time: 312 seconds total
  Average test duration: 13 seconds
  Flaky tests: 0 detected

ATTACHMENTS UPLOADED:
  • test-failure-3d-secure.png (screenshot of popup state)
  • payment-flow-coverage-report.json
  • ci-build-log-excerpt.txt (for context)

COMPREHENSIVE TEST MATRIX:

  BROWSER COVERAGE (each test run on 3 browsers):
    Chrome 125: 24/24 passing
    Safari 17: 22/24 passing (2 skipped, Safari-specific issues)
    Firefox 124: 24/24 passing

  ENVIRONMENT COVERAGE:
    Staging: Full run (28 tests) → 23/24 passing
    Production simulation: 20/24 passing (3D Secure, Apple Pay skipped)

AUTOMATION STATUS:
  ✓ GitHub Actions trigger: Configured (runs on every PR to main)
  ✓ TestRail API connection: Validated
  ✓ Test result schema: Mapped correctly
  ✓ Failure notifications: Slack alerts enabled
  ✓ Auto-assign: Features enabled for assigned people

SLACK NOTIFICATIONS SENT:
  Payment flow test run completed: 20 passed, 2 failed, 2 skipped
  Threshold alert: Enabled if pass rate < 80% (currently 83%)
  Assigned users notified: @marcus, @priya (failed test owners)

COVERAGE ANALYSIS:

  Critical paths covered:
    ✓ Happy path (form filled → payment → confirmation)
    ✓ Error handling (invalid card, expired card, insufficient funds)
    ✓ Edge cases (high amount, international card, declined charge)
    ✓ Post-payment (invoice generation, email, receipt)

  Partial coverage:
    ~ Alternative payment methods (Apple Pay, Google Pay, Klarna)
    ~ Mobile checkout flow (viewport < 768px)

  Gaps identified:
    ✗ Network failure during payment (timeout scenario)
    ✗ Multiple simultaneous payment attempts (race condition)
    ✗ Accessibility audit for payment form

RECOMMENDED NEXT TESTS:
  1. Add 3 tests for failed payment recovery scenarios
  2. Add mobile-specific checkout flow tests (Cypress mobile mode)
  3. Add accessibility audit test (axe-core)

HISTORY & TRENDS:
  Previous runs (last 7 days):
    • Build #846: 21/24 passing (87%)
    • Build #845: 20/24 passing (83%)
    • Build #844: 22/24 passing (92%)
  Trend: Stable, slight dip on builds 845-847 (3D Secure issue introduced)

REMEDIATION STEPS:
  1. Review 3D Secure test (retest with 5s wait, watch manually)
  2. Deploy gift-card-regex-fix PR
  3. Re-run payment flow suite (expect 24/24 passing)
  4. Update TestRail run status to COMPLETED

DOCUMENTATION:
  • Test case descriptions linked in TestRail (27/28 updated)
  • CI configuration committed to repo (.github/workflows/payment-tests.yml)
  • Local run instructions added to README

NEXT SYNC:
  Automatic: Every commit to payment-flow-tests branch
  Manual: Anytime via npm run testrail:sync
  Schedule: Daily full suite run at 2am UTC

ACCESS & SHARING:
  TestRail report: https://testrail.example.com/run/12345
  Share link: Public (expires in 7 days)
  Assigned team members: @marcus, @priya (notified)
  Dashboard widget: Payment flow results pinned to team dashboard

About this skill

Your CI run passed, but TestRail still shows yesterday's results. Or every renamed and parameterized test creates another case. This skill reconciles test identities, scopes the correct run, and handles retries and incomplete reports before publishing results.

What it does

  • Matches automated tests to TestRail cases using explicit IDs and stable automation keys, with ambiguous matches held for review.
  • Produces a case diff covering additions, renames, field updates, and tests absent from the current inventory.
  • Parses JUnit-style results into logical tests and attempts, preserving parameter values and execution environments.
  • Builds runs for the intended case IDs and maps outcomes against the project's actual status definitions.
  • Prevents blind resubmission after timeouts by reconciling remote result history and recording batch receipts.
  • Reports submitted, verified, skipped, unresolved, and unmapped results separately, with failure evidence where supported.

What you get

A targeted reconciliation report and, when access and authorization are available, updated TestRail cases and verified run results. Without access, you get a provisional mapping and concrete payloads or integration code with unresolved decisions clearly marked.

Best for

QA engineers and developers connecting CI or local automated suites to TestRail, cleaning up case drift, or investigating why release test reports disagree with execution results.

SKILL.md preview

SKILL.md
---
name: testrail-case-and-ci-result-sync
description: Reconcile automated tests with TestRail cases and publish CI results using stable identity matching, scoped runs, project-specific status mappings, and readback verification. Use when the user says "sync CI results to TestRail", "create TestRail cases from these tests", "my sync keeps creating duplicate cases", "publish this JUnit report", "TestRail disagrees with CI", "reconcile renamed tests", or "report release test coverage".
---

# TestRail Case and CI Result Sync

… (sign up to view the full skill)
Sign up to view, copy, and install the full skill

More development skills

View all Development skills →

React and Next.js Regression Test Coverage

Generate React and Next.js tests with Jest, Testing Library, MSW, and Playwright, prioritized by regression risk.

Development

Playwright E2E Tests and Flake Repair

Build reliable Playwright tests, repair flakes from evidence, migrate suites, and configure faster, diagnosable CI.

Development

Pre-Ship Verification for Code Changes

Before calling a code change done, Claude grades every claim by evidence, tries to break it, and reports what's verified vs assumed.

Development

Claude Skill Validation and Quality Scoring

Audit SKILL.md structure, test supplied scripts, and get an evidence-backed grade, quality tier, and prioritized fixes.

Development