There seems to be two conflicting needs in software testing in healthcare: traceability and test performance.
Traceability
Traceability is the ability to link your requirements to your tests:
In healthcare, regulatory compliance is on the side of traceability. Reasonably enough, regulators want to know that products actually do what they’re designed to do. Traceability provides that chain of evidence.
Test Performance
However, traceability can come at the expense of test performance — defined as how efficiently a product can be tested both for defects and for requirements conformance. Increased test performance is based on automating the right tests and also merging a number of test cases into test scenarios that focus not just on product functionality but on appropriate workflow. In many industries, the practice is to group several requirements and test them all at once through a few test scenarios. But healthcare regulators (and regulatory compliance owners in vendor organizations) tend to avoid this practice. They want to see proof of code coverage and requirements in test plans. Also, regulators are focused on making sure features are implemented as designed. In this context, there is less emphasis on bug-hunting — although as a rule, vendors and regulators both agree bug-catching is critical.
Add Agile to the Mix
Many software development organizations within healthcare are dipping their toes into Agile methodologies and hoping to gain from reduced project risk, iterative customer feedback, and more cohesive teamwork. But traceability can be more of a challenge with Agile than waterfall methodologies. In Agile, Quality Assurance (QA) tests during each sprint, as the team delivers new features. But there’s no way to get rid of the integration testing bottleneck at the end, where the whole product needs to be tested.
Is A Traceability Matrix The Answer?
One way to address the traceability-performance dilemma is a traceability matrix. Owned by QA, a traceability matrix links requirements, functionalities, workflows, and tests to ensure the entire product is being tested. If you use Word or Excel for traceability documentation, you’re guaranteed to add more documentation overhead. In these formats, traceability matrices are time-consuming and complex to create and maintain. The result is that many organizations don’t automatically include a traceability matrix in in their documentation plans. And you might even find some Agile proponents arguing against traceability matrices as non-Agile.
But there are real advantages to starting and maintaining a traceability matrix. And if you set up your requirements, functionalities, workflows, test cases, and test scenarios in a database instead of a Word document or an Excel spreadsheet, you can simplify maintenance exponentially. All you’d need to do is keep each element up to date. Then generate the matrix with a simple SQL query.
What do you think? Do you have other ideas on maintaining traceability without sacrificing test performance? Share your thoughts in the comments.