Turning Ad-Hoc AI Chat Into a Reusable Test Design Skill

 Turning Ad-Hoc AI Chat Into a Reusable Test Design Skill

Most teams already ask AI for test ideas. The usual flow looks like this: paste a ticket, get a long list of cases, copy what looks useful, then do it again next week with a slightly different prompt — and get a slightly different shape of output.

That works once. It does not scale.

What changed for me was not “use a smarter model.” It was encoding my test design rules into a Cursor skill so the same workflow runs the same way every time.

The problem

Ticket comments often already contain happy-path scenarios, regression notes, and edge cases someone thought of in review. What they rarely contain is a ready-to-execute plan: concrete test data, numbered steps, expected results, and a clear split between “does the product still work?” and “can someone abuse this?”

Without a shared format, every engineer invents their own table, depth, and security coverage. Reviews become about formatting, not risk.

The idea: teach the agent once

Cursor skills are markdown playbooks. You create them with the built-in create-skill flow: describe the task, answer a few setup questions, and land a SKILL.md the agent follows when that workflow is attached or invoked.

I did not ask for “write some tests.” I defined a pipeline:

• Read a ticket (or a pasted QA comment).
• Extract explicit test / regression scenarios.
• Produce about 10 regression and 10 security cases.
• Require test data and execution steps on every row.

Show the plan in chat, then optionally post it back to the same ticket as a human-readable comment.

That last step mattered: the deliverable lives where the team already works, not only in a chat thread.

What went into the skill

• A useful skill is specific about shape, not just intent.

Output columns (kept lean):

ex:

ID | Title | Test data | Execution steps | Expected result | Notes

I dropped fields like “maps to” and “priority” after the first iteration — they looked complete but slowed writing and were unused in practice.

Hard limits: ~10 regression, ~10 security. Caps force prioritisation and stop the model from padding with weak duplicates.

Security lens (ticket-aware): authn/authz, injection/XSS, open redirects, IDOR, sensitive data exposure, session issues, unsafe URL handling — only where the change is relevant.

Guardrails:

• Do not invent product behaviour that contradicts the ticket; mark assumptions in Notes.
• Stay plan-level unless someone asks for automation code.
• Never put API tokens or secrets in the skill file or in ticket comments.
• Credentials come from the environment (or a one-time session ask), not from the playbook body.

Those rules are what make “AI for testing” feel like a teammate with a checklist, not a random idea generator.

How a run looks in practice

Input: a ticket URL or a comment that lists scenarios for a feature change.

Agent work:

• Resolve the issue and pull description / focused comment.
• Parse named scenarios and note gaps (missing data, unclear steps).
• Fill regression slots from labeled regression notes and high-value happy paths.
• Fill security slots from realistic abuse of that surface.
• Render tables in chat for quick review.
• Post the same structure back to the ticket when approved.

Example regression case (generic):

• Title: Multi-line free text keeps value after blur and continue
• Test data: Staging funnel; sample paragraph with line breaks (non-sensitive content)
• Steps: Reach the field → enter text → blur → continue → return if possible → confirm value
• Expected: Value persisted; no layout break; continue enabled when required

Example security case (generic):

• Title: Script payload in free text is stored/rendered safely
• Test data: "><script>alert(1)</script> in a non-production environment
• Steps: Enter payload → save/continue → reopen view that displays the field
• Expected: No script execution; content escaped or sanitised per product rules

The important part is not these two examples — it is that every case is forced into the same executable shape.

Iteration beats the first prompt

The skill was not perfect on day one.

v1: Broad functional + many NFR buckets

v2: Focus only on regression + security (~10 each)

v3: Require test data + execution steps; drop unused columns

v4: Auto-post a readable comment to the same ticket

That is the real pattern: ship a skill, run it on a real ticket, tighten the playbook from friction, then reuse.

Why this works better than “just chat”

Ad-hoc chat: prompt drifts every time, easy to forget security, output dies in the thread, hard to onboard teammates.

Skill-backed workflow: same columns, limits, and rules; security is a required section; optional write-back to the ticket; share or attach the skill.

AI still needs judgement — thin tickets still need parent context or clarifying questions. The skill does not replace QA thinking; it standardises the packaging of that thinking.

How others can reuse this approach

You do not need my product or tracker. The pattern is portable:

Pick one painful, repeatable QA task — e.g. “ticket scenarios → regression + security cases,” or “PR description → smoke checklist.”

Use create-skill and answer the boring questions up front — personal vs project skill, when to attach it, table vs prose, what buckets to include.

Write the workflow as steps, not vibes — inputs → parse → derive with caps → output template → optional integration (Jira, GitHub, Linear).

Define the table columns you will actually execute — if a column is never used in a test session, delete it.

Keep secrets out of the skill — base URL defaults are fine; tokens stay in env vars.

Run on three real tickets and revise — the third run will expose padding, missing steps, or over-broad security fluff.

Add write-back only after the chat plan is trusted — posting bad cases to a ticket is worse than not posting.

A minimal skill description might read:

Turns ticket comments that list test and regression scenarios into about 10 regression and 10 security cases with test data and execution steps, then posts that plan back to the same ticket.

That one sentence is enough for the agent to know when to use it.

What this is not

• Not a replacement for exploratory testing or domain experts.
• Not automatic Cypress/Playwright generation (unless you extend the skill later).
• Not a license to paste production data into prompts — use staging and sanitized fixtures.

Treat the skill as a contract between you and the model: same inputs, same kind of outputs, same place the team can review them.

Takeaway

• Using AI for testing gets useful when you stop treating every ticket as a new conversation and start treating test design as a productized workflow.

• Create-skill gave me a durable place to put that workflow. The generated cases were only as good as the rules I wrote — and those rules got better every time I ran the skill on a real change.

• If you try one thing: encode your team’s “good test case” definition once, attach it when you plan from a ticket, and iterate the skill instead of reinventing the prompt.