Quickly Extracting a Relevant Message from Megabytes of Data

Home » Tutorials » HL7 Message Filtration and Representation » Quickly Extracting a Relevant Message from Megabytes of Data

Introduction

Suppose that you’ve created a filter for finding a particular message in a file using the Search and Filter tool from Caristix Workgroup. Now, you want to re-use this filter for multiple message sets. However, you don’t want to go through the normal overhead of loading the message set in Search and Filter, and then reloading the filtered messages when your filter is applied.

To bypass this overhead, you can use the Workgroup Console, a Command Line Interface that allows you to run Caristix Workgroup tools. This will allow you to more rapidly process large message files.

In this scenario, you’re going to learn the basics of using the Workgroup Console to rapidly perform a rapid Search and Filter operation. You’ll first set up a basic filter ruleset using the Search and Filter tool. This filter will be used to find any ADT_A01 – Admit/visit notification messages for a patient named Mary Smith. Following that, you’re going to use the Command Line to use that filter on a file containing thousands of HL7v2.6 messages and output any matching messages to another HL7 file.

Creating the Filter

To create a filter, open the Search and Filter tool from Caristix Workgroup.

As stated above, you are looking for an ADT_A01 message where the patient Mary Smith is the subject patient. To obtain only ADT_A01 messages, you need to create Data Filters based on the message structure as detailed in the MSH – Message Header segment. To obtain only messages relating to Mary Smith, you need to create a Data Filter relating to the PID – Patient Identification segment.

Creating the MSH Data Filters

To ensure that only ADT_A01 messages are accepted by the filter, you need to create Data Filters on the MSH.9 – Message Type field. The first component of that field is the Message Code, and the second is the second is the Trigger Event. For an ADT_A01 message, those components are ADT and A01 respectively. Thus, you need to create a Data Filter for each of those components such that the components have their respective desired values.

The first Data Filter is on the MSH segment, field 9, component 1, and the desired value (using the “=” operator) is ADT.

The second Data Filter is also on the MSH segment and field 9, but is on component 2 and the desired value is A01.

Using these filters, only ADT_A01 messages will be present in the output messages.

Creating the PID Data Filter

Now that you can capture ADT_A01 messages only, you want to ensure that all ADT_A01 messages are related to the patient Mary Smith. Thus, you want to create a filter based on the patient’s unique ID, targeting the PID.2 – Patient ID field, component 1 – Id Number. For the data you’ll be using for this tutorial, the patient’s ID is 2219922122. Therefore, you need to add a Data Filter on the PID segment, field 2, component 1, where the value is equal to 2219922122.

With this, you’ve created a full ruleset for finding the desired message. Now, you need to save these Search and Filter Rules onto your computer’s file system – not the Caristix Library. Save them under the name “Search and Filter CLI Tutorial Rules.”

Executing the Search and Filter Command

The basic structure of the Search and Filter command is as follows:

$ Search-And-Filter-HL7 <file(s) to filter> -sfr [ <optional flags> ]

The -sfr flag indicates the location of the Search and Filter rules. For the purposes of this tutorial, the optional -r flag, which denotes the location of the output file(s), will be used as well to indicate where the output file where be created.

Tip: you can view the full list of options for the Search And Filter command by typing in Search-And-Filter-HL7 help.</div

For the file to filter, download this sample message set which contains thousands of HL7 messages including one ADT_A01 message pertaining to Mary Smith. Now, run the command. Assuming that all the relevant files are in your user folder, the command should look something like this:

$ Search-And-Filter-HL7 "C:\Users\MyUserName\Sample.txt" -sfr "C:\Users\MyUserName\Search and Filter CLI Tutorial Rules.cxf" -r "C:\Users\MyUserName\results.hl7"

The resulting file, results.hl7, will contain a single message: an ADT_A01 – Admit/visit notification for the patient Mary Smith.

You’ve now successfully created a Search and Filter ruleset and used it in the Workgroup Console. You can use this technique to process very large files – gigabytes’ worth of HL7 messages – without having to go through the lengthy process of actually loading them into the Search and Filter tool.

Download

You can download the Search and Filter ruleset that was created during this tutorial by clicking here.