No formal testing effort is complete until a comprehensive Test Summary
Report has been written and approved. Often validation teams think of a Test
Summary Report as one last documentation insult to their precious time and
energy. (p.94)
Weary testing teams sometimes argue
that they have a formal requirements document, a Test Plan, a Trace Matrix and
a huge number of Test Scripts and Result Logs, surely that should be enough
proof that they worked hard to test the system. Working hard is not the issue.
Working smart and in compliance is the issue. Validation teams need to think of
their Test Summary Report (TSR) as the billboard by which they sell management,
auditors, and inspectors on the excellent job they did in formal testing of a
regulated system to assure its reliability in performing its intended function
and compliance to GXP.
The Test Summary Report provides the
opportunity to scope testing effort:
·
Who were the testers, witnesses, and script
writers - what were their names?
·
What were the dates of the testing?
·
Where were the testing sites, e.g., all at one
site, multiple sites from other countries, from employee remote VPN logins,
from client or partner sites...?
·
Were there deviations from the Test Plan with
Test Scripts or Test Sites added or deleted and why?
·
What were the issues arising and how were they
resolved during testing?
·
When was each task listed in the Test Plan
completed?
·
How many requirements were tested and does the
Trace Matrix show that each requirement was tested by at least one Test Script?
·
Which Test Scripts challenged 21 CFR Part 11
and EU GMP Annex 11 requirements for the system with negative testing actions?
·
Were all system modules/functions tested? If
not, why not?
·
Provide a table to identify each failed Test
Script with a description of why each failed, how it was repaired, and the
retest results after the Fix cycle.
·
What tools were used for issue tracking, Test
Script creation, execution, and Results management?
·
What is the recommendation to Management for
release of the system for GXP production use based on the testing?
A comprehensive Test Summary Report
along with examples of some failed Test Scripts, the Trace Matrix, and Approved
Test Plan and Requirements document should be enough to satisfy an auditor or
inspector without spending hours scouring through test results. Without a comprehensive
Test Summary Report, an auditor or inspector could spend hours going through
test results and will invariably find something amiss from results written in
pencil to skipped or missing printouts, etc.
Next Month: What is a Platform
Requirements Specification (PRS)?
Separate from the URS, a PRS should be defined to describe the users
operational requirements, e.g., 24/7 access by telephony, the software
supplier’s technical requirements, e.g., database and infrastructure specifications,
and IT operational needs, e.g., fit with other applications, as well as ongoing
support of the software application on other systems in the organization. (p.153)
