Unstructured Data

Sensitive data can be found in unstructured data (free text) such as clinician notes or other narrative text. Most of the data within an unstructured field is not sensitive, but there are times when it might contain data elements you want to protect.

This generator will replace any piece of information found in another message field that is set for de-identification.

Example #1

In the following message, the name of the patient is mentioned in the patient update note (NTE.3).

MSH|^~&|SYSTEM-A|1|||20100404210829||ADT^A08|20100404000000645509|P|2.3|||||CA|ASCII
PID|0001|ID53572812^^^|0126271^^^^^1||SMITH^JOHN||195307280000|M|SMITH^JOHN||1 FIFTH AVENUE^NEW YORK^NEW YORK^^33333^USA^P^53052^16||(555)555-5555|(555)555-5555|^|2|||238898464|||||||||C1||N
NTE|||Mr. Smith provided new phone numbers


If the patient name (PID.5.1 field) is listed among the de-identification rules, you can configure a new field to detect the patient name within NTE.3

Segment Field Component Subcomponent ID Generator
PID 5 – Patient Name 1 – Family Name     Excel File
NTE 3 – Comment       Unstructured Data

 

Using these settings, the de-identified message will look like this:

MSH|^~&|SYSTEM-A|1|||20100404210829||ADT^A08|20100404000000645509|P|2.3|||||CA|ASCII
PID|0001|ID53572812^^^|0126271^^^^^1||Doe^JOHN||195307280000|M|SMITH^JOHN||1 FIFTH AVENUE^NEW YORK^NEW YORK^^33333^USA^P^53052^16||(555)555-5555|(555)555-5555|^|2|||238898464|||||||||C1||N
NTE|||Mr Doe provided new phone numbers

 

Example #2

MSH|^~&|SYSTEM-A|1|||20100404210829||ADT^A08|20100404000000645509|P|2.3|||||CA|ASCII
PID|0001|ID53572812^^^|0126271^^^^^1||SMITH^JOHN||195307280000|M|SMITH^JOHN||1 FIFTH AVENUE^NEW YORK^NEW YORK^^33333^USA^P^53052^16||(555)555-5555|(555)555-5555|^|2|||238898464|||||||||C1||N
NTE|||Mr Smith ( ID53572812 ) provided new phone numbers
NTE|||Mr Smith also provided a new address

If the patient name (PID.2 field) is listed among the de-identification rules, you can configure a new field to detect the patient ID within NTE.3

Segment Field Component Subcomponent ID Generator
PID 2 – Patient ID       Numeric
PID 5 – Patient Name 1 – Family Name     Excel File
NTE 3 – Comment       Unstructured Data

 

Using these settings, the de-identified message will look like this:

MSH|^~&|SYSTEM-A|1|||20100404210829||ADT^A08|20100404000000645509|P|2.3|||||CA|ASCII
PID|0001|123459876^^^|0126271^^^^^1||Doe^JOHN||195307280000|M|SMITH^JOHN||1 FIFTH AVENUE^NEW YORK^NEW YORK^^33333^USA^P^53052^16||(555)555-5555|(555)555-5555|^|2|||238898464|||||||||C1||N
NTE|||Mr Doe (123459876) provided new phone numbers
NTE|||Mr Doe also provided a new address

 

Example #3

Sometimes, a field may be Base64-encoded, as seen below.

MSH|^~&|SYSTEM-A|1|||20100404210829||ADT^A08|20100404000000645509|P|2.3|||||CA|ASCII
PID|0001|ID53572812^^^|0126271^^^^^1||SMITH^JOHN||195307280000|M|SMITH^JOHN||1 FIFTH AVENUE^NEW YORK^NEW YORK^^33333^USA^P^53052^16||(555)555-5555|(555)555-5555|^|2|||238898464|||||||||C1||N
NTE|||TXIuIFNtaXRoIHByb3ZpZGVkIG5ldyBwaG9uZSBudW1iZXJz


In the above message, the decoded value of the NTE.3 field is “Mr. Smith provided new phone numbers”. To detect and de-identify the patient’s name, in addition to including the patient name (PID.5.1 field) into the de-identification rules, you need to tick the “Decode message field from base 64 format before De-Identifying” checkbox. This will decode the field, de-identify it, and then re-encode it into Base64.

Segment Field Component Subcomponent ID Generator
PID 5 – Patient Name 1 – Family Name     Excel File
NTE 3 – Comment       Unstructured Data

 

Using these settings, the de-identified message will look like this:

MSH|^~&|SYSTEM-A|1|||20100404210829||ADT^A08|20100404000000645509|P|2.3|||||CA|ASCII
PID|0001|ID53572812^^^|0126271^^^^^1||Doe^JOHN||195307280000|M|SMITH^JOHN||1 FIFTH AVENUE^NEW YORK^NEW YORK^^33333^USA^P^53052^16||(555)555-5555|(555)555-5555|^|2|||238898464|||||||||C1||N
NTE|||TXIuIERvZSBwcm92aWRlZCBuZXcgcGhvbmUgbnVtYmVycw==

 

The decoded value of the above NTE.3 field is “Mr. Doe provided new phone numbers.”