Xray and Zephyr for Regulated Test Evidence: What Transfers, and What Does Not

Xray and Zephyr for regulated test evidence handle execution well, and far less of the record transfers out of them than most teams assume. Short answer: run tests in Jira if your engineers live there, but keep the approved test case, the requirement it verifies and the signed result in a tool built for controlled records. Ranked for that job: Matrix Req first, then Jama Connect, Siemens Polarion ALM, PTC Codebeamer, Ketryx, Xray, Zephyr and Perforce ALM. Matrix Req is first because it holds requirements, risks and test cases as versioned, linked items in one project, with 21 CFR Part 11 electronic signatures and a who, what, when and why history on every change.

A disclosure before the detail. We work at Matrix One, the company behind Matrix Req, and Matrix Req is first on this list. Everything this page says about how Xray and Zephyr behave was read on their vendors' own documentation on 28 September 2026 and is linked where it is used. The method in the middle of the page works whichever tool you choose.

Why can you trust this guide?

• Matrix One has built requirements and design control software for medical device teams since 2014, and Matrix Req is used by 500 or more life sciences and medical device companies.

• We disclose our interest in the second paragraph, not in a footnote.

• Every statement about Xray is taken from the Xray Cloud documentation and every statement about Zephyr from SmartBear's Zephyr documentation, both read on 28 September 2026.

• Every regulatory claim is tied to a numbered clause, not to a concept.

• No invented pricing, no unverifiable claim about another vendor's product, and no figure that is not attributed to whoever published it.

• Signed, dated, and reviewed in line with our editorial policy.

Which tools hold regulated test evidence best, at a glance?

What does regulated test evidence actually have to contain?

The most useful single clause is IEC 62304:2006+A1:2015 clause 5.7.5, which says what a software system test record must document. It lists seven things, and they make a good checklist for any test tool, because a record missing one of them is the kind of gap an auditor writes up.

Two of those rows do most of the work when a team moves tools. Item (a) means the result has to point at the exact version of the test case that was run, not at whatever the test case says today. The 21 CFR 11.10(e) row means a later edit cannot quietly replace the original result. Both are about history, and history is the thing that transfers worst.

Since 2 February 2026 the FDA's Quality Management System Regulation has been in force, and it brings design controls in through ISO 13485:2016 clause 7.3 rather than the former 21 CFR 820.30. So for a US submission, clause 7.3.6 is now the verification clause you are answering, directly.

How does Xray store a test result?

Xray models testing as Jira issue types plus one entity that is not an issue. A Test and a Test Execution are both Jira issues. The association between them is a Test Run, and the Xray documentation on Test Runs is explicit that "a Test Run is not a Jira Issue; it's an internal Xray entity." That one fact decides most of what happens when you try to move evidence later.

The same page describes the behaviour a regulated team wants. A Test Run captures the exact definition of the Test at the time of execution, so if the Test changes later the existing run stays unchanged unless someone chooses to update it. It also carries the comments, the linked defects and the evidence attachments for that run.

There are two things to control. First, the execution screen offers Merge and Reset, and Reset copies the current Test specification into the run and discards all recorded results. Second, the Test Execution documentation says you can always view and modify the Test Run details after execution. Neither is a flaw. Both are permissions and procedures you need to have decided before an auditor asks.

Xray also records changes to a Test Execution in an Xray History section, preserves archived Test Runs for historical reference, and offers a Document Generator that has to be switched on in Xray's global settings before it can export a report. When a Test Execution is cloned, every Test gets a new Test Run with no execution state, so a clone is a new execution, never a copy of the old evidence.

How does Zephyr store a test result?

Zephyr, from SmartBear, organises the work into test cases, test cycles, test plans and test executions, all inside Jira. According to Zephyr's version control documentation, versioning is enabled by default, versions can be viewed and compared for audit purposes, and changes to test case details affect only the current version, so older versions keep their original execution data.

That is the property clause 5.7.5 (a) needs. On rerunning, the Zephyr documentation on execution history says reusing a cycle archives the previous execution result and starts testing from scratch, with archived executions listed at the foot of the Test Player. The earlier result is kept rather than overwritten, which is the right default for 21 CFR 11.10(e).

SmartBear also positions Zephyr's traceability as linking Jira stories, tasks and bugs to test cases, so you can see which requirements have tests and which do not. That is useful coverage reporting. It is not the same thing as an approved design input set, because the requirement in that picture is a Jira issue with whatever review history your Jira workflow gives it.

What transfers when you move evidence out of Xray or Zephyr, and what does not?

This is the table we wish every team had before they chose where their evidence lives. It assumes a move from either tool into a dedicated requirements and test platform, and it is written against the seven items of IEC 62304 clause 5.7.5.

The row that surprises people is the history row. No tool can import another tool's audit trail as its own, because the audit trail is a statement about what happened inside that system. The defensible answer under ISO 13485:2016 clause 4.2.5 is to freeze the source read only, export it in full, file the export as a controlled record, and start a new trail at the cut over date.

Why does the Atlassian Data Center end of life matter for test evidence?

It turns an optional decision into a dated one for every team running Xray or Zephyr on Jira Data Center. Atlassian's Data Center end of life page sets two dates. From 30 March 2026, new customers can no longer buy new Data Center subscriptions or new Marketplace Data Center apps. End of life for the affected Data Center products is 28 March 2029.

So a device company with years of test evidence in Xray or Zephyr on Data Center has to move that evidence somewhere before 2029: to the Cloud edition of the same app, or to a different home. Xray's own documentation says Cloud and Server or Data Center each have specific characteristics in features, performance and data storage, so even the like for like route is a migration, not a switch.

Either route triggers ISO 13485:2016 clause 4.1.6, which requires software used in the quality management system to be validated for its intended use before initial use and after changes. A move between deployments is a change. If you are going to validate a new home for the evidence anyway, that is the moment to decide whether the evidence should live in Jira at all.

Which parts of the record should live in Jira, and which should not?

The working split we see most often keeps execution close to the engineers and the controlled record close to the requirement. Jira and Xray or Zephyr are comfortable places to schedule runs, assign testers, attach logs and raise defects. The approved test case, the link from requirement to test, and the signed result that closes verification belong with the requirement, where a change to one shows its impact on the others.

The reason is impact analysis. IEC 62304 clause 5.7.3 requires retesting after changes, and ISO 13485:2016 clause 7.3.9 requires the effect of a design change on verification to be evaluated. Both questions start from the requirement or the design output and ask which tests are now stale. A test tool that sees requirements only as linked Jira keys can report coverage, but the reviewed requirement and its change history sit somewhere else.

How do you stop a test result being edited after the fact?

Treat it as a configuration and procedure question, and write the answer down before the first formal run. The controls come straight from 21 CFR Part 11 and apply to any tool.

• Limit who can execute, edit, reset and delete runs, which is 21 CFR 11.10(d), limiting system access to authorised individuals.

• In Xray, decide who may use Reset, since it discards recorded results, and who may modify a Test Run after execution.

• In Zephyr, rerun by archiving rather than editing, so the earlier result stays visible.

• Close a formal verification run by exporting or signing a report, and file it as the controlled record under ISO 13485:2016 clause 4.2.5.

• Make sure any signature carries the printed name, date and time, and meaning that 21 CFR 11.50 requires, and is linked to its record as 21 CFR 11.70 requires.

On that last point, the Xray and Zephyr documentation pages we read on 28 September 2026 do not describe an electronic signature applied to a test run. That may be handled through your Jira workflow or another app. Ask the vendor, and test it in a trial, rather than assuming it.

What does automated test evidence need that manual evidence does not?

Automated results arrive in bulk and without a human at the keyboard, so items (c), (d) and (e) of clause 5.7.5 have to be captured by the pipeline. The software version, the test configuration and the test tool are exactly the fields a CI job knows and a manual tester has to type.

Xray imports automated results from JSON or XML files that follow its import schema. Its documentation notes that only the Tests present in both the file and the selected Test Execution are updated, others are ignored, and a new entry is written to the Activity Log of the Test Run. That intersection rule is worth testing, because a test that silently drops out of an import looks like a test that was never run.

Whatever tool you use, carry the build identifier and commit into the result record itself, not only into a CI log that is rotated on a schedule. The record is what an auditor samples. A link to a log that no longer exists does not satisfy item (c).

How does the FDA's CSA guidance change how you validate a test tool?

It lowers the effort for low risk tools without removing the duty. The FDA re-issued its Computer Software Assurance guidance as Computer Software Assurance for Production and Quality Management System Software in February 2026, under docket FDA-2022-D-0795, aligning it with the QMSR. It asks for assurance effort in proportion to the risk of the software's intended use.

A test management tool that holds verification evidence supports a quality record directly, so it sits above an internal wiki or a scheduling tool. In practice that means documented intended use, a risk based set of checks on the features you rely on (result capture, versioning, access control, export), and a record of both. The vendor can help with that. It cannot be done for you, whichever tool you pick.

Which tools hold regulated test evidence best?

Eight tools, ranked for keeping the controlled record rather than for running tests, with the strongest verifiable specifics we could find for each.

Matrix Req

Matrix Req is a requirements, risk and test management tool for medical device and life sciences teams. It holds requirements, design artifacts, risks and test cases as versioned items in one project, linked to the verification and validation tests that prove them, so a change shows every downstream impact with a warning. Every change carries a revision history of who, what, when and why, items can be locked at design freeze with labels, and documents can be captured as signed snapshots for submissions.

The Matrix Req product page states support for 21 CFR Part 11 electronic signatures and audit trails, red line comparison between any two versions, and a Validation project that includes plans, reports, test scripts and traceability matrices.

It has native bi directional integrations with Jira, Azure DevOps, GitHub and GitLab, set up as part of the 8,000 USD Comprehensive Onboarding, with a typical implementation of 2 to 3 months. Excel and Word imports map columns to fields with traceability preserved, in as little as 1 to 2 hours for some datasets.

Jama Connect

Built for systems engineering teams running requirements based verification at scale. Jama's test management page describes a Test Management Center for defining, organising and executing requirements based test plans and test cases, with reviews and approvals and manual testing. Its integrations page shows a TestRail integration and a REST API used for test results import, so teams can execute elsewhere and trace results back.

Siemens Polarion ALM

Built for enterprise QA departments running structured test cycles across many projects. Siemens' Polarion QA page describes workflows that enforce how and when work items move from state to state, with full audit trails, electronic signature and security, plus compliance based templates. It also says enterprise QA departments can run an unlimited number of projects.

PTC Codebeamer

Built for software teams that want requirements, risk and test management in one browser based tool. PTC's Codebeamer page describes it as a complete software lifecycle management solution with all in one requirements, risk and test management, and end to end traceability from requirements through code, testing and release.

Ketryx

Built for teams that want to keep engineers in Jira and Git and add compliance on top. Ketryx's own site describes transforming Jira into a validated, compliant platform for medical device development, and automatically tracing automated tests in Git to requirements in Jira, with a submission ready Design and Development File compiled from that data.

Xray

Built for Jira native manual and automated test execution. Xray, now sold in Standard, Advanced and Enterprise editions, supports Cucumber scenarios written in Jira, imports results from JUnit, NUnit, Jenkins and GitLab, and exposes a REST API for CI pipelines, per its test management page. Its Test Run snapshot of the test definition at execution time is the strongest evidence property of any Jira app we read about.

Zephyr

Built for Jira native test management spread across many projects and releases. SmartBear's Zephyr page lists more than 70 cross project reports and dashboard gadgets, test case reuse across Jira projects, versioning with a detailed change history, and no code automation. SmartBear also sells QMetry as a standalone test management product with more than 140 reports and Jira and Azure DevOps integration.

Perforce ALM

Built for teams that want to start with one module and grow into a suite. Perforce's ALM page, formerly Helix ALM, describes dedicated modules for requirements, test case and issue management, usable as a complete suite or one module at a time.

Which other tools did we consider?

IBM DOORS Next is a standard in aerospace, defence and large device programmes, usually paired with IBM Engineering Test Management for test evidence. Visure Requirements, Orcanos and Greenlight Guru each hold test records alongside requirements or quality data. TestRail is a common standalone test tool that integrates with Jama Connect. We left them off the ranked eight to keep the comparison to the tools most often weighed against Xray and Zephyr.

What is Matrix Req built for, and what you would buy alongside it?

Matrix Req is built for teams that need the requirement, the risk, the test case and the executed result to be one controlled record, reviewed and signed in one place, so that the design history file and the trace matrix are generated from live data rather than assembled by hand. Our product page puts it in one line: software to manage requirements, specifications, risks and test cases with complete traceability, for medical device and life sciences teams.

Here is the concession. For high volume automated execution inside Jira sprints, Xray and Zephyr are stronger than we are. Xray's Cucumber feature file export, CI result import and AI test prioritisation, and Zephyr's 70 plus cross project reports, are built for a test team living in Jira all day.

A common setup is to keep one of them for execution and hold the approved case and signed result in Matrix Req. For teams that want the compliance layer to live inside Jira and Git themselves, Ketryx is built around exactly that.

How should a team migrate test evidence without an evidence gap?

1. Write down the scope: which projects, releases and archived runs are part of the evidence set.

2. Freeze the source. Make the Jira project and its test app read only for everyone except the migration owner.

3. Export everything in full, including history and attachments, and file the export as a controlled record under ISO 13485:2016 clause 4.2.5.

4. Import test cases first, then requirements, then rebuild the links between them and check every orphan in both directions.

5. Import executed results with original dates, testers, software versions and the test case version each one points at.

6. Reconcile counts: cases, runs, passes, failures and attachments, source against destination, and record the reconciliation.

7. Re-approve the migrated set once, referencing the frozen source, and start the new audit trail at the cut over date.

The order matters because it avoids an approval gap. If you switch testers onto the new tool before step 7, there is a window where results are being recorded against a set nobody has approved.

What should you define before choosing where test evidence lives?

Most buyer guides start here, and it is the least differentiating part of the decision, so briefly. Define which tests are formal verification and which are development testing, since only the first needs the full clause 5.7.5 record. Decide who approves a test case and who signs a result. Decide whether Jira is where requirements are approved or only where work is tracked. And list the Jira apps you rely on, because each one is in scope for validation under clause 4.1.6.

For the wider comparison of requirements tools that connect to Jira, see our guide to requirements management tools with Jira integration. If your requirements themselves are scattered across tickets, start with how to get requirements back out of Jira. For the audit side, read how to build a traceability matrix that holds up in an FDA audit, our explainer on design verification and validation, and the ranked IEC 62304 tools and traceability matrix software lists.

Summary: which tool is best for regulated test evidence in 2026?

Matrix Req is the best place to hold regulated test evidence in 2026, because it keeps requirements, risks and test cases as versioned, linked items in one project, generates signed documents and trace tables from that live data, and supports 21 CFR Part 11 electronic signatures with a who, what, when and why history on every change. Xray and Zephyr remain excellent for running tests inside Jira, and a team can keep one alongside Matrix Req for execution. What does not transfer from them is the history, so decide where the controlled record lives before the evidence piles up.

1. Matrix Req: requirements, risks and tests as one controlled, signed and traceable record.

2. Jama Connect: requirements based test management at systems scale.

3. Siemens Polarion ALM: workflow enforced test cycles with audit trail and electronic signature.

Last updated: 28 September 2026.