What’s the Glue Holding EHR Migration and Conversion Projects Together?

Caristix is sponsoring a series of blogs posts on John Lynn’s Hospital EMR and EHR blog. You can read the latest post, What’s the Glue Holding EHR Migration and Conversion Projects Together? here: http://www.hospitalemrandehr.com/2015/08/13/whats-the-glue-holding-ehr-migration-and-conversion-projects-together-optimize-healthcare-integration-series/

We’re focusing on the particular issues of QA. What are the tools you need to support the work of your QA teams?

HIEs and Interface Testing

Interoperability barriers and HIE business models

HIEs and interface testingIn a previous post, we talked about the interoperability barriers facing HIEs (healthcare information exchanges). What are they? Primarily 1) cost and 2) complexity. That is, the cost of interface development, and the technical difficulty involved. These are especially in context of the business model switch, as HIEs move from startup grant-based funding to self-generated revenue based on membership fees and services.

The problem is, self-generated revenue is tricky to achieve at best. In some cases, the fees barely cover costs. So how do smart HIE leaders make it work? For one thing, they aren’t focused on cutting headcount (because in many cases, these are lean operations), but they are focused on speed and quality. They drive down cost by increasing speed to interface delivery without sacrificing quality.

Why requirements are critical

They increase speed and quality both by ensuring that their interfacing teams focus on requirements. Not just any requirements – the right requirements. The right interfacing requirements, gathered at the right time, set the stage for clearer communication and expectations.

Read more about HL7 requirements.

How to reduce costs while increasing speed and quality

Good requirements are one side of the interface delivery coin. The other side: testing. Once HIE leaders have put in place a strong requirements culture, they tackle interface testing and validation, which is the process of ensuring that the interface works per the requirements.

Read more about what can wrong with HL7 interface validation. It applies to HIEs and their interface testing.

HIE leaders focus on testing and validation because the interfaces they deliver must broker key workflows between partners: hospitals, physician practices, labs, immunization registries, cancer registries, and more. The good thing is that many workflows and requirements are standard (or near-standard) across sites. The bad thing is that data structure and data content will vary.
How do you know you’re hitting your requirements? Testing and validation. Most HIEs do this. The problem is, small changes to an interface can take a long time to test and validate. For instance, a 15-minute coding change can take 2 weeks to validate.

The best way to reduce time spent on testing and validation is to automate the process. So that 2-week test cycle can be reduced to 1 hour.

And that’s exactly what smart HIE leaders are doing: they are finding ways to automate testing. This ensures that interfaces are delivered faster – and with more quality and confidence.

Caristix and HIE interface testing

The Caristix platform automates testing and validation for interfaces. Check out our tutorials for common validation scenarios.

Watch the on-demand demo of the Caristix Workgroup platform. You won’t be disappointed.

What is HL7 Interface Validation?

What is validationThe purpose of HL7 interface validation

In the interface lifecycle, validation is critical. HL7 interface validation is about checking the interface you’ve developed: making sure it works and that it meets the requirements you captured before interface development.

The most important requirements are around clinical workflow. In essence, validation is about testing against your workflows. You should test performance, message volume, and edge use cases, all before the applications and interface go live.

Need more detail on what you should test? Get an HL7 interfacing checklist here.  

What can go wrong?

There are two ways HL7 interface validation can go wrong.

First, if you’re pressed for time, you might be tempted to cut corners and skimp on validation. This is a bad idea because go-live will be a disaster. Without checking workflow, edge cases, and performance, you are setting yourself up for downtime and frustrated clinicians who can’t access the data they need. Avoid this problem by documenting a validation plan based on your requirements, at the start of the project.

Second –ironically enough — good validation practices can put you in a trial-and-error, break-fix nightmare. We’ve worked with teams that faced this issue. Because they were thorough testers, a 30-minute code update required 2 weeks of testing and validation to ensure the interface met requirements. This is the danger with manual validation practices.

But you can solve this problem with automated HL7 interface validation and testing.

3 reasons you should automate validation

  1. Run validations over and over again. Once you set up the scenarios, you can repeat them endlessly. So that 3-week trial-and-error cycle can be compressed to a few hours.
  2. Generate the test data you need. When you automate, you get to cover both common and uncommon workflows, work with plain-vanilla (expected) and weird-and-wonderful (unexpected) data and patient information, just one message or thousands of them.
  3. You’ll gain accuracy. With automation, there is no temptation to skip validation scenarios. You also have a record of what was tested. The record is a by-product of the validation process, so there is no extra work to get this information.


Validation How-Tos

The Caristix platform automates HL7 interface validation, enabling you to set up test scenarios and run them as often as you need. We’ve put together a list of tutorials and how-tos for common validation scenarios. Check them out. You can use them as-is with Caristix software, or adapt them to your own test environment.

See HL7 interface validation tutorials.

HL7 Interface Testing Checklist

hl7-testing-checklistOver the past few weeks, we’ve reviewed multiple HL7 testing topics. So you know how critical it is to test interfaces during interface configuration, the validation phase, and during maintenance.   This HL7 interface testing checklist will help you design a testing process that covers your most important needs. And if you already have a testing process in place, it will help you identify any areas of concern.

1. We’re running tests to make sure we’re not injecting errors during interface development, configuration or during maintenance.
This type of test is called regression testing. It’s important to do this as you make major and minor changes to the interface.

 2. We’re positive our tests cover our interoperability requirements because:

  • We confirmed our interface engine handles our standard workflows.
  • We test each system independently to make sure they work separately.
  • We verifying our edge cases anticipating unexpected values.

We test network infrastructure so we know how we impact overall infrastructure performance.

3. Our clinicians trust our validation process knowing they’ll have consistent data flow in their clinical applications.

If they have concerns, you need to build in checkpoints the testing process.

4. We run the most critical tests across all interfaces for all sites.

Document your most critical tests, and document that you’ve run the tests.  

5. We’re testing with de-identified sample messages drawn from production data in order to cover our requirements.

Production data is realistic and will cover your use cases during testing. But it’s critical to remove PHI.

6. We’re testing in an independent test environment using the same configuration as our production systems.

Don’t use your production environment for testing. The risks are too great.

7. We document our testing process completely to ensure traceability and repeatability.

Keep this information accessible.

8. Testing takes us longer than any other part of the HL7 interface lifecycle.
If you checked this statement, you should consider test automation. Test automation will enable you to speed up testing and increase your test coverage.

 9. Our test reports include the number of times a test was run and duration, test results, the messages used and a summary of the test scenarios used.

How efficient is your testing process? Is it time to automate your testing?

See how Caristix technology automates testing. Check out this 2-minute excerpt on interface testing and validation from our on-demand demo. See how to prevent costly project rework and delays.

HL7 Test Reporting

Tip 16 in the Interoperability Tip Series

The past few tips have covered HL7 testing and test automation. This one covers what you should do with HL7 test reporting.

Creating Test Reports    

Caristix Test Module

As part of the testing process, you’ll want to run reports. The reports should document the following: 

  • The number of times the test was run, as well as test duration – if you’re sending messages, this helps you understand performance.
  • Test results, including positive validations and failures.
  • The messages that were used; note the data source (SQL queries pulling from a database, an HL7 message feed, a batch file).
  • Summary of test scenarios that were run.

Just as you need sample messages elsewhere in the interfacing lifecycle (for instance, scoping), you need them for testing, too.

As a reminder, sample messages give you custom formats, structure, and data values.

For example, imagine you’re interfacing the lab information system and the ADT. You’ll want to look at the issues that were highlighted during the gap analysis. Your test messages need to cover your use cases and the following:

  • Events that are exchanged
  • Code sets/vocabulary and varying field lengths
  • Optional segments and fields, especially varying optionality  

Go with Production Data

Now about the sources of your messages: we’re going to come right out and say it – you need production data. Here’s why: Once you have the right message samples and test messages, you need to make sure you have a sufficient volume of quality test data. And your production data accurately reflects the data you work with day in and day out, both in data type and format as well as volume. And that means you’ll be able to accurately test for load and performance, and avoid message workflow problems that can bring down interfaces.

In our next tip, we’ll explain how you can de-identify production data to use safely in your test system.

The Caristix 3.1 release focuses on test automation. These new features add a lot of power to the Caristix platform. To get started on using them, check out our new test automation tutorial series.

Create a Healthcare Data Test Environment

Tip 15 in the Interoperability Tip Series

Test environment

Once you’ve developed a test plan and test scenarios, you need to configure your interface in a test environment.

Healthcare Data test vs. production

What do we mean by test environment? Essentially, another instance of your interface engine, along with simulations of the clinical systems you’ll be interfacing.

It’s important that you do your testing in a test environment, not in a production environment. It’s easy to think it can’t hurt to test in a live system, but here are three reasons why that’s a big mistake:

  • If you forget to cancel or delete all test transactions once you’re through with testing, you’ll end up with faulty transactions in your production systems.
  • You run the risk of impacting ePHI or HIPAA-protected health data.
  • You don’t want phantom data turning up in a CMS audit. Your clinical systems contain data that constitute a legal record.

So what’s the right way to go about it? Set up your test system using the same configuration as your production system, including the same rights and versions (it’s OK if IP addresses are different). Make sure you upload enough patient data, and that your tests cover your requirements (we can’t say that often enough).

Learn more about test automation

Want to see how Caristix technology automates testing? Check out this 2-minute excerpt on interface testing and validation  from our on-demand demo. See how to prevent costly project rework and delays.


 Image credit: r.nial bradshaw

Data Workflows and Interface Testing

Tip 14 in the Interoperability Tip Series

interface testing and data workflows

In last week’s tip, we talked about capturing workflows.

Here’s why. Before you can conduct any  interface testing, you must understand what to expect of your workflows. This should include common workflows – such as a patient being transferred – involving the use of the products that will be interfaced.

For example, in many hospitals, emergency department and in-patient ADTs are two separate systems. A new patient that comes through the emergency department would be registered in the ED’s ADT first. And if she is transferred to Med/Surg, you would need to populate the main ADT, either through an interface or manually by re-entering the data.

Or if you’re creating an interface to move patient charge data from a surgical information system to a billing system, you would need test scenarios in which:

  • Patient demographics and patient ID are incomplete.
  • Billing item information is incomplete.

Interfacing workflows: normal use cases and edge cases

And in fact, this is why is you need to test normal use cases as well as edge cases – where the data is incomplete, or otherwise deviates from the norm.

Interface testing: is the engine behaving?

With that understanding in place, you can test to make sure the interface engine behaves as expected for standard – as well as unexpected – workflows. When it comes to edge cases, you’ll need to consider more possibilities.

 For example, if your interface engine does not accept a certain range or type of data, you’ll need to send such data to it – e.g., a date of birth of 1850 or entered in reverse – and see if the interface triggers an error.

Does the code have new errors when you correct a previous bug?

Are you introducing errors during testing? You’re testing the data format and confirming that you’re not introducing errors.

When you code an interface, your specification will be based at least in part on sample messages. By definition, you know that these messages work. So don’t use only these sample messages in your texts.

The danger of limited test messages in interface testing

Let’s say your test patient in your sample messages is called John Smith – with four characters in the first name. You test your interface using these sample messages, and everything works. But three months from now, your hospital admits a patient named Benjamin O’Donnell, only no one tested for 8 characters in the first name and an apostrophe in the last name. The interface doesn’t like it, and you have a support call (and a none-too-happy clinician) to handle.

This is where test automation comes in

By automating your testing, you will feel freer to test at any time and you’ll be more confident about making changes because you’ll know you can easily test each time you change the interface as you’re coding.

Learn more about test automation

Want to see how Caristix technology automates testing? Check out this 2-minute excerpt on interface testing and validation  from our on-demand demo. See how to prevent costly project rework and delays.

Image credit:  TheTurfBurner, Creative Commons licensed for reuse

HL7 Test Automation: Where’s the Low-Hanging Fruit?

HL7 Test Automation: Low Hanging FruitTesting and validation are important tasks, as we explained in the Interoperability Tip Series and the HL7 Survival Guide.

Shouldn’t interfacing be easy by now?

Testing is what takes the longest when you’re building an interface. So when you hear about interface engines that allow you to get an interface into production in a couple of hours, that’s absolutely true. Coding is quick with modern interface engines. But bear in mind that coding time seldom includes testing. We’ve worked with organizations that can code a change in 15 minutes, but need 2 weeks to actually implement — due to testing.

Given the need to test, test automation can save you a lot of time.

Why Automate Testing?

Simple: the benefits are clear. Organizations that adopt test automation spend less time on project lifecycles. They meet tighter deadlines with less project effort. They uncover more bugs, before end-users are impacted. The code is easier to maintain. In a nutshell, when it comes to software testing, it’s all about “find early, fix cheaply.”. Interfaces are no different; after all, they are simply forms of code.

Why Automate HL7 Testing? 6 Key Reasons

1. Save time – the most obvious benefit.

2. Repeatability. Set a series of tests. Run them as often as needed, without manual intervention. Validate that you didn’t break anything simply by pushing a button. Apart from saving time, you get consistency. If you’re a vendor, you ensure that the most critical tests are run across all sites, regardless of client availability. So you deliver consistent quality. The result is less downtime and increased client satisfaction at lower cost.

3. Avoid frozen interface syndrome, maintain interfaces more easily.

4. Implement changes much faster, so clinicians gain access to newer functionality and information in source and destination systems. Ultimately, this puts you on the path to easier, more agile interoperability.

5. Ensure traceability. Test reports allow you to document that tests succeeded and/or failed over time.

6. Increase test coverage. More testing works out to better quality. Manual testing takes so much effort that you can’t test and retest everything. You end up focusing on just the highest risk items.

What If You’re Busy?

These are great reasons to embark on test automation. The organizations that embark on automation get it. But there are other teams that are so crazy-busy, with their heads practically underwater, that they can’t add any more to their plates.

We get that. New methodologies and new ways to get things done put you on a learning curve. Even if the payoff is worth it, it can be hard to get started.

What if there was low-hanging fruit? There is now. We’ve put together some basic tests/validations that are tedious to run manually but easy to automate. If you’re looking for the low-hanging fruit of HL7 test automation, start with segment and field validation here:

Validate Field1 = value
Validate Field1 = Field2 within the same message
Validate Field is X characters long
Validate Field contains a limited set of characters
Validate Field does not contain a set of characters
Validate Field is a valid date

These tests will get you going. They’re easily set up and run easily. Adapt them to your own test automation software. Or if you want to get HL7-specific, ask for an intro to Caristix software to check them out. They’re quick to set up and run repeatedly.

Interface Test Types

Tip 13 in the Interoperability Tip Series

Last week in Tip 12, we covered when, why, and what to test when you’re working with interfaces. This week, we’re looking at the different interface test types a team needs to perform.

Make sure that your tests cover your interoperability requirements. These will vary depending on the systems you’re working with. Be sure to also cover the following:

1. Workflow

Confirm the interface engine handles your standard workflows as expected.  Just as a reminder, workflows are the series of messages (ADT, lab orders, lab results, etc.) that reflect information flow. You might have dozens, depending on the complexity of the systems and patient care scenarios in your hospital or client site.

2. Edge cases: unexpected values

If you’re testing birth dates, include 1899 as well as 2017. Include dates with the month and day reversed. Try different abbreviations for the days of the week. Check all caps on names. Check accented names. Check hyphenated last names (Lowe-Smith), and those with an apostrophe (O’Donnell). They’re more common than we think, and they can trip up an interface, especially those with customized delimiters.

3. Performance, load, and network testing

Though interface developers don’t normally test network infrastructure, you may want to do this during the validation phase to see how workflows and data are impacting overall infrastructure performance. A high-volume interface may need more load testing than a low-volume interface, depending on your interface engine and connectivity infrastructure.

4. Individual systems

You should test each system on its own, kind of analogous to unit testing in software development. For instance, in addition to making sure the surgical and billing systems handle workflow end to end, make sure they work separately.

Learn More about Test Automation

Want to see how Caristix technology automates testing? Check out this 2-minute excerpt on interface testing and validation from our on-demand demo. See how to prevent costly project rework and delays.


Healthcare Interface Testing: When, Why, and What to Test

Tip 12 in the Interoperability Tip Series

healthcare interface testingIn tips from the past few weeks, we covered two requirements-related artifacts analysts must create: 1) profiles or specs and 2) gap analyses, which include mapping tables.

In this tip, we look at testing an interface. And see how it doesn’t have to be an exercise in frustration.

When to Test: 3 Phases

You need to perform healthcare interface testing at 3 different phases in the HL7 interface lifecycle: during configuration and development; during the formal validation phase; and during maintenance.

Why Test An Interface?

When you start to develop and iterate on your interface, you run tests to avoid introducing new problems – you check and test your code to make sure you are not injecting errors. This is true both during interface development or configuration and while in maintenance mode. This testing helps you determine whether or not the interface makes sense and meets your requirements.

Once you’re satisfied with the interface, you move to validation testing. This is when you determine if the interface will work with and meet the requirements of your clinical workflow. Specifically, you test performance, extreme data cases, and how well the interface supports large volumes. By figuring this out before go-live, you save a lot of implementation headaches, and alleviate the time clinicians need to spend helping you validate the interface once you go live.

What Matters: Reduce the Cycle Time

Healthcare interface testing can be time-consuming. Some organizations find that testing activity is the most time-consuming part of interface development. George, a Caristix client who works for a healthcare vendor, explains, “The implementation process for our product runs several months. Throughout that timeline, we have checkpoints where we need to make sure the feeds are sending data that our product can consume. First, when we scope the feeds, we have to make sure we document all the gaps and work out with the customer how to bridge them. Then the feeds get built. And we test them, and find more gaps. We fix those gaps, and then test, and find more. The cycle repeats as needed until all gaps are resolved.”

That’s the essence of testing.

The Next Step: What to Look for in a Test Tool

So that’s the what, when, and why of testing. But how do you handle testing efficiently? The key is to automate HL7 interface tests.  While you need to spend time during the development/configuration phase setting up the tests, during the validation phase, you can take advantage of automation and save a lot of time. Of course, Caristix offers test automation. Some interface engines include test tools. Regardless of the source of your test software, make sure you can do the following:

  • Be able to connect to web services or a database, such as by calling a web service, and check in the database after sending a message
  • Validate inbound and outbound messages
  • Validate ack and nack
  • Generate values and test messages from a profile or specification, and generate a large volume of data/messages if you’re conducting volume testing
  • Repeat test plans/scenarios, and create reports

We’ll cover more on testing in the upcoming weeks. Stay tuned for guidance on test types, test reports, and test systems.

Learn More about Test Automation

Want to see how Caristix technology automates testing? Check out this 2-minute excerpt on interface testing and validation  from our on-demand demo. See how to prevent costly project rework and delays.