In this post we will be examining EDI ASC X12 documents. The ASC X12 electronic data interchange format is much older and more widely used than formats like JSON or XML. The scope and scale of the EDI format is positively massive. Many of the transactions across a huge number of businesses and government agencies are conducted via the exchange of EDI documents. In many cases the ASC X12 EDI format is the industry standard. This post will start by looking at some of the general definitions and concepts related to these documents. We will then look more in depth at the specific structure and syntax of the EDI format and discuss some considerations when designing an application that uses these documents for data interchange.
Understanding EDI ASC X12 Documents
Overview
In simplest terms, the EDI ASC X12 format is a set of standards and rules that define a certain syntax for structuring and transferring data between two or more parties. In this sense, these documents are analogous to better known formats like JSON and XML. The general ASC X12 standard consists of several different versions and many different sub-standards. The general ASC X12 format has gone through several major updates/revisions over the years resulting in different versions. Some common versions in use today include 004010, 005010 and 006010. More meaningful than the version are the many different sub-standards. These sub-standards are generally the principal way in which EDI documents are identified. Each sub-standard generally corresponds to a specific industry or field. For example, there are EDI documents used to transmit data in finance, shipping and insurance. A three-digit number is used to identify these sub-standards. The 856 document for example is used to represent shipping notices and/or manifests. The 837 document is used to submit payment for health care claims. Nowadays, version 005010 is standard across the health care field. Therefore, to be complete in our designation, if we were working with health care claims we would be using the 005010 837 document. Each sub-standard like the 856 or 837 defines its own unique rules and structure in addition to what is defined by the general ASC X12 version format. You can go to the link below to see a list of many different 004010 ASC X12 EDI documents.
List of X12 documents for EDI version 4, Release 1 (004010)
The general structure and syntax of EDI documents
At the most basic level EDI documents are conceptually similar to XML and other data formats. In almost any data format you have some way of defining a group or unit of data and you express the structure of the overall data document via some “nesting” scheme. In XML a grouping of data is defined by tags and these tags are nested inside each other. EDI documents are similar in that they are essentially a nested collection of groups of data. XML is very easy to read and understand because each separation and grouping of data is explicit and written out. You always know exactly when one grouping or level of data has ended because of the presence of a closing tag. Unfortunately for EDI documents things are not so simple. Often the overall structure is not explicitly written out but rather implied in other ways. We will describe this in more detail going forward. For now, let’s look at the basic units of data in an EDI document.
In an EDI document all data is written out as “segments” and “elements”. A segment is made up of an identifier and a finite number of elements. Each element is just a piece of data, for example a number or a string. An element may be a composite element, consisting of several pieces of data known as “components”. You can think of a composite element as an array. Segments and elements are separated by characters which serve as delimiters. Unless otherwise specified, the element delimiter is denoted by the ‘*’ character and the segment delimiter is denoted by the ‘~’ character. The delimiter for the components of a composite element is the ‘:’ character. All of the actual text within an EDI document will consist entirely of “segments” and “elements”, all other structure and meaning is implied by the relative location of the segments and elements. Below is an example of one segment from an 837 document.
CLM*A101*37.7***11:B:1*Y*A*Y*Y***********~
The very first part of the segment is the identifier. The identifier is not an element; it merely identifies the segment. We can see from the above that this segment is a “CLM” segment. This segment consists of 20 elements each separated by the ‘*’ character. We see that the first element is the string “A101”. The second element is the decimal 37.7. Elements 3 and 4 are empty and have no value. Element 5 is a composite element with three components, the number 11, the character ‘B’ and the number 1. Element 9 is the character ‘Y’ and elements 10-20 are all blank. We see that the “CLM” segment has ended by the presence of the ‘~’ character.
The next question is, what do these values mean? From just looking at the above there is absolutely no way to tell. Each EDI document has an associated guide/dictionary that specifies the meaning of each segment and each element. This guide is necessary for an EDI document to have any meaning. If I wish to read the above segment I need a copy of the guide to read from. If I am designing an EDI document parser, a copy of the language rules expressed in the guide must somehow be written into my program. Unfortunately, these guides are only provided via the ASC X12 standards committee and must be purchased. So for example if I needed the 005010 837 guide I would purchase it below.
Purchase the 005010 837 Implementation Guide
If you clicked the above link, you’ll notice that the 837 standard is further broken down into 837-P, 837-I and 837-D. Let’s assume we are working with the 837-P document. From referencing the corresponding guide, I find that the “CLM” segment specifies claim information. In other words, this segment represents a health care claim that I am submitting for payment. The second element of this segment specifies the monetary amount of the claim. So we see that I am submitting a claim for the amount of 37.7 dollars. Each other data element is similarly defined in the official implementation guide. The guide also contains additional information such as what kind of data is allowed for each element. For instance, some elements are only allowed to have a decimal value, others a string value of a certain length. In some cases, there are actually only a fixed number of possibilities that are allowed for an element. For instance, the guide may define that the only allowable values for an element are ‘A’, ‘B’ and ‘C’. Most EDI guides will also contain some information on general business concerns and industry specific information.
Headers
Beyond segments and elements almost all other structure in an EDI document is implied, usually by the relative position/location of segments. All EDI documents start with a number of headers that are sometimes referred to as the “envelope”. These headers are just special segments that are at the beginning of every EDI document. There are usually three such headers, the interchange control header, functional group header and transaction set header. These are some of the only segments that have special matching end segments. Generally, the only indication that a segment has ended is the presence of the ‘~’ character. Let’s look at an 005010 837-P document with only the necessary headers. Note that I will be using placeholders here for the actual element values. You will need to look at the implementation guide for the correct values, the idea here is to help understand the structure.
ISA*1*2*3*4~
GS*A*B*C~
ST*1*2*3~
SE*55*66~
GE*a*b~
IEA*11*22~
The ISA segment is the interchange control header. The GS segment is the functional group header and the ST segment is the transaction set header. The SE (interchange control trailer) segment corresponds to the ST segment and indicates an end of the transaction set. Similarly, the GE segment corresponds to GS and the IEA segment corresponds to the ISA segment.
Loops
The final major structural pieces of an EDI documents are known as “Loops”. A Loop in this sense has nothing to do with a programmatic loop. In an EDI document a “Loop” is simply a grouping of segments. A “Loop” may be repeated but it does not need to be. In an 837-P document one of the most important loops is Loop 2300. This loop contains all of the segments necessary to list all of the information regarding a single claim. Loop 2300 always starts with one “CLM” segment. We saw an example of this segment earlier. Loops can be further divided into sub loops. Not all segments need to be contained in a sub loop. Loop 2300 for example consists of a number of segments and several sub-loops each themselves consisting of a number of segments. Sub loops are usually denoted by the parent loop followed by a letter. Loop 2300A and Loop 2300B for example are sub loops within Loop 2300.
One tricky thing about loops is that there are no distinct segments that indicate the start or end of a loop. We can only know a loop has started or ended by looking at the segments around it. Loop 2300 is nice since from the guide we know it will always begin with a “CLM” segment. However, in most cases it is much trickier. Some Loops start with the same segment. For example, in an 005010 837-P Document Loops 1000A and Loops 1000B both start with an NM1 segment which indicates a name. There are many other cases where the structure can become quite difficult to infer. For example, many Loops within an EDI document are optional. Therefore, the ending of one Loop can be marked by multiple different possible segments at multiple different possible places. Thus in most cases, parsing or writing an EDI document requires a complete programmatic implementation of all the rules and syntax defined in the official guide. One cannot simply traverse the document according to the presence of starting and closing tags.
Hierarchical Segments
Sometimes the loose, implied structure described above is insufficient. Many EDI documents including the 837-P we have been using in our examples require additional information so that a fixed structure can be achieved where necessary. This is accomplished by adding so-called “Hierarchical Segments”. These segments are included solely to indicate that the current loop is acting as a parent or child loop. So we have some places where the structure is explicitly defined for us via these hierarchical segments and some places where the structure is not explicitly defined. Contrast this to a format like XML where the structure is inherent to the format itself. Here’s an example of what a hierarchical segment would look like in an 837-P document.
HL*3*1*22*0~
Exactly how these “Hierarchical” segments are implemented is again dependent on what kind of EDI document we are working with. In an 837-P document the first element 3 is a unique index shared across all HL segments. The second element indicates that this Loop is a child of the Loop with the HL segment where the index is 1. The third element is a code which identifies what kind of data this loop contains. The fourth element indicates that this Loop has no other child loops with Hierarchical segments. Remember that these hierarchical segments do not appear in every Loop. Some Loops have them and some do not.
Let’s look at another example 837-P document containing more loops and segments. Note that I am omitting the data elements for each segment and have not included all necessary segments for a valid 837-P document. I will be using comments indicated by the ‘#’ character to mark the start of Loops and I will be using indentation to indicate the full structure. Note that these comments are not valid EDI markup, they would need to be removed.
ISA*1*2*3*4~
GS*A*B*C~
ST*1*2*3~
# Loop 1000A
NM1~
PER~
# Loop 1000B
NM1~
Loop 2000A
HL~
PRV~
# Loop 2010AA
NM1~
N3~
N4~
REF~
# Loop 2000B
HL~
SBR~
# Loop 2000BA
NM1~
# Loop 2010BB
NM1~
#Loop 2300
CLM~
CLM~
SE*55*66~
GE*a*b~
IEA*11*22~
In reality this example document would most likely appear as follows
ISA*1*2*3*4~GS*A*B*C~ST*1*2*3~NM1~PER~NM1~HL~PRV~NM1~N3~N4~REF~HL~SBR~NM1~NM1~CLM~CLM~SE*55*66~GE*a*b~IEA*11*22~
Using EDI Documents In Your Application
Now that we have some understanding about the structure of EDI documents let’s talk about how to integrate this data format into our applications. Most applications will be concerned with transferring data in the form of EDI documents back and forth between your application and some other external third party. There are some things you need to be aware of before going forward. You will typically need a business partner or trading partner agreement in place with the third party you are exchanging data with. You will also need to agree upon how the EDI documents will be transferred back and forth. In other words, what communication software/protocol will you use? Next, see if the third party you are working with maintains their own EDI implementation guide. Almost all large third parties that exchange data via EDI documents will have their own implementation guide. First and foremost, this implementation guide should specify what values to use for certain elements. For example, you will need to know what name and information to use for the receiver name loop 1000B in an 005010 837-P document. The implementation guide will also most likely include additional information about how to structure your document and what segments/elements to use. The general EDI format is somewhat flexible. There are many optional loops and segments that you can choose to send or not depending on your business needs. Furthermore, the same data can often be sent using different combinations of loops and segments. There are many allowable variations that would all result in a “compliant” ASC X12 EDI document. Therefore, it is up the businesses/groups exchanging data to agree upon their own fixed standards.
Once you have everything in place with your trading/business partner the next step is to determine how your application will translate data from your system to the EDI format. In the best case scenario there will already some agreed upon commercial software in place to parse and write EDI documents. In this case you simply need to pass your data into this software and then transmit the resulting documents. If there is no agreed upon translating software in place, we should still avoid writing our own parser unless absolutely necessary. The EDI format is simply too complex with too many different rules and edge cases across many different standards. The amount of time you will spend attempting to parse even just a small subset of possible documents is almost never justified. Rather you should look for existing solutions and build on them to suite your needs. For instance, when working with a Rails application try to find a well established gem. One such gem that has worked well for many ruby developers is stupidedi. However you end up implementing your EDI translator, it is imperative that this part of your application have very good test coverage. Between the many different kinds of EDI documents and the specific needs and rules set up by individual third parties, there is almost never a simple one size fits all solution here.
Hopefully this post has helped you to understand the various EDI document formats on a general level. Remember that this post is not and cannot be a comprehensive guide. The vast majority of the implementation details will depend on what kind of EDI document you are using and the third party or parties that you are working with. You will need the general EDI guide/dictionary for your document version and type as well as the specific implementation guides maintained by your trading/business partners.