Fog Creek Software
Discussion Board




Is EDI still a hot topic?

I'm a couple of years out of the field, and I was wondering if manufacturers are still hot for EDI. If they are, what new technology has come to play in the last couple of years? Is everybody moving to XML transactions for EDI? Are manufacturers moving from ANSI formats to others, such as EDIFACT?

Clay Dowling
Friday, August 08, 2003

Was EDI a recent hot topic?  I've done a bit of work in the last year or so with EDI systems, without even knowing that EDI was some sort of standard for data interchange. 

I just thought the systems I was interfacing with were legacy systems with outdated modes of interaction with the outside world!  I did come to appreciate it a bit more, after I worked with them a bit, but I'm surprised it was ever a hot topic.

Herbert Sitz
Friday, August 08, 2003

1) EDI is the standard for HIPAA interchanges, which are now mandated by federal law for all medical data transfer. [this is highly simplified, but to give you an idea of the demand]

2) Any business system older than three years that talks to other companies will likely use EDI

EDI is *everywhere*, but it's in the areas that most programmers don't see. The reason you don't hear more about it is that everyone, Microsoft included, seems to think that by pushing XML, that makes it the de facto standard. Problem is the thousands of companies that have spent six or seven figures on an EDI-based business system don't get the same level of excitement over change for the sake of change.

So, in summation - EDI is *not* a hot topic in the IT press, but for the billions of dollars in transactions that move around every day, it's a very hot topic.

Philo

Philo
Friday, August 08, 2003

Much thanks for the update.  In answer to the questions, it was indeed a hot topic among manufacturers a few years ago. I made a very nice living programming EDI systems a few years back.

I hadn't been aware of the development on the hospital front. I've never programmed to the HL7 standard (which is what hospitals use), but it didn't look significantly different than the ANSI documents used for electronic commerce.

Myself I wouldn't mind seeing the change to XML for interactions, provided a common DTD was agreed upon. With the old ANSI documents it was hard to get agreement between different divisions within the same company.

Clay Dowling
Friday, August 08, 2003

"Myself I wouldn't mind seeing the change to XML for interactions, provided a common DTD was agreed upon"

XSD. Schemas are the new DTD's and (IMHO) better.

There are some movements to create an XML standard for EDI documents. One that I saw was incredibly messy - the group in charge seemed to think that every XML element had to be individually unique, so they were huge. Tags looked like:
InvoiceInvoiceItemServicesServiceItemServiceCharge

Which I find unreadable. They seemed to miss the point that XML *can* be contextual.

Philo

Philo
Friday, August 08, 2003

EDI is huge in a business sense but not sexy in a tech sense. It is ecommerce before the internet revolution. I remember reading about how XML was going to change EDI transactions in 97 or 98. The XML/EDI group (or SGML) has been around for just as long and its mail archives from 97 are nice to read.

Tom Vu
Friday, August 08, 2003

"EDI is huge in a business sense but not sexy in a tech sense"

Do you mean as in "nobody's writing about it" or "it's no fun to work with"? Because I take issue with the second statement - I think it's fascinating and I have a great time playing with it.

Heck, how often do we get national standards to back up our work?

Philo

Philo
Friday, August 08, 2003

I'm curious, are there any tools out there that simplify working with EDI?  The only two times I've worked with it I've written the code from scratch.  (Well, at least the first time was all from scratch; the second was a modification of the first.)  Anyway, it seems like there ought to be a somewhat easier and faster way.

Herbert Sitz
Friday, August 08, 2003

We certainly don't get national standards for automotive EDI.  Each of the auto companies have their own standard. I'm suprised to find somebody who really enjoys working with EDI.

I'm technically competent with it as well, but at times it gets "interesting" to be the liason between the supplier and automaker. Some of that interesting relationship is why I dropped out of doing EDI work for a while.

Just out of curiosity, Philo, what platform do you work on?  Most EDI job postings I'm seeing right now seem to be using AS/400, SAP or both. I used to work with UNIX and NT.

Clay Dowling
Friday, August 08, 2003

Right now I'm nursemaiding a custom system I built in C# against SQL Server. Next on the list is a biztalk implementation for a different group.

Even with the automaker specs - YOU HAVE A SPEC. I've got the full X12 guide online here. "How many characters in this field?" check the spec - 30. That's the answer. On the other hand is the wonderful world of business applications where specs aren't written and answers change daily.

I'll take a spec, thanks. :-)

Philo

Philo
Friday, August 08, 2003

From my experience EDI is not hot, but still in use, mostly in big companies. It costs them money to move to XML, that process is very slow.

Evgeny /Javadesk/
Saturday, August 09, 2003

Gah.  On the topic of ANSI X12 271 (a health insurance format being used for HIPAA compliance), all I can say is "GAH!"  That was the most annoying data format I have ever seen in my life.  It was fractally annoying -- every little bit you look at is just as annoying as the whole thing.  And who ever heard of a hierarchical format without end tags?  The only reason I didn't tear my hair out was that I buzz my head!

Yes, Philo, you do have a spec, but at least with your typical flat-file layouts it's conceivable that a user could remap the fields.  And in Access (yes, I implemented an X12 reader in Access 97 -- don't ask), it's just one line of code.

Thus far, that's been my first experience with EDI, and I sincerely hope it's my last.  But (desperately attempting to turn this into an on-topic response rather than just a vent) there's definitely work to be had in EDI.  After all, if the spec were simple enough that users could figure out how to import it on their own, what would all those EDI programmers do for a living?  ;>

Sam Livingston-Gray
Sunday, August 10, 2003

EDI spec annoying?

When you're trading stocks or sell car parts, you don't want the users to redefine the formats!  That's why it works!

EDI is easy.  In my experience, all concerned parties define a set of messages.  The messages consist of items of data, the format of these is set, as Philo above hinted at.

You may need a quantity field.  It gets defined as Numeric, 12 digits, no decimals.  It gets a Name - Quantity, and a Code.

First you define all of the fields needed in your messages, like so.  Fields 1-5.

1 - Name, X30
2 - Colour, X6
3 - Part#, 999999
4 - Quantity, 999999999999
5 - Weight, 999.999

The format of the messages themselves seems complicated, but it's really simple.  There is a bitmapped header, followed by ASCII data. 

A 'Buy' message may have 3 fields, Part#, Colour and Quantity.  That means we'll be needing fields 2,3 & 4.  Out bitmap header is then 01110000.  (ie 1 byte).

The rest of the data gets tacked on.  (X is the bitmap)

Message = "X Green000123000000000001."

Here we are ordering 1 of Part 123, in Green.  Data is padded, so no need for delimiters, length bytes and such.

The other fun bit is that fields can be optional, in this case colour could be optional.  To leave it out, just turn the bit off, and leave out the data.

To decode, check the bit map (ah, 2nd bit is on, that's Colour, so take the first 6 bytes).  Repeat.

EDI is a neat system, even if it's not sexy (generally mainframe & non-internet - wot's X25?)

AJS
Sunday, August 10, 2003

AJS,

X.12 is the ANSI standard that most of us have to program to, unless we're lucky and get edifact (the UN standard, heavily used in Europe).  The format you're describing looks nothing like ANSI X.12. 

We don't get a bitmapped header, and we don't have fixed length fields. We get a file with two mystery delimiters that we have to figure out by reading the document.  Because the actual EDI document is likely to be enclosed by data specific to the transmission protocol (and a single company is likely to use multiple transmission protocols to communicate with it's various customes), it's moderately entertaining to figure out exactly what the delimiting characters are.  They're in positions 4 and 106 of the fixed length header.  But it's possible for the transmission data to look just like the header, so your code has to double check things, like make sure the appropriate data follows the header.

As bad as that sounds, I found it pretty easy to do in C and C++. It's less easy with other languages, such as the Progress 4GL, and it wasn't much more fun with Visual Basic.

To be honest, the standards aren't the hard part, and they don't lead to any real stress.  The hard part is that EDI shows up any weakness in business practices. A customer with 100 trucks sitting in his receiving lot gets very testy when the shipping clerk couldn't be bothered to record the trailer number they sent the parts on.  The place it shows up is the EDI document, and the first person to hear about it is going to be the fellow who put in the new EDI system.

Clay Dowling
Sunday, August 10, 2003

"They're in positions 4 and 106 of the fixed length header."

Heh. I just burned an EDI tools company on this. You should be checking 4, 106, *and* the first character after the first ">"  (Hint: if you're validating EDI and just checking 4 & 106, then one bad character in the envelope is going to void the entire document)

Philo

Philo
Sunday, August 10, 2003

"InvoiceInvoiceItemServicesServiceItemServiceCharge"

That is unmistakably XCBL.  My fellow developers and I have had the pleasure of burying all of that under an API that maps the fields into a nice java object so you can just say "item.getCharge()", and such.

And yes, we have discovered that most of our business partners are still in EDI, which is translated into XCBL for us by a third party to whom we pay money.

Jim Rankin
Monday, August 11, 2003

I will say, though, that XML is a Better Way to create data formats than "check char 4 and char 106 for the delimiter" type formats.  XML is much less fragile than that, and any added verbosity can be taken care of by Moore's law and decent compression algorithms.

And the silly redundancies of standards like XCBL are in no way XML's fault.

Jim Rankin
Monday, August 11, 2003

Philo,

That's the sub-element delimiter you're talking about, right?  If it is, I never had to deal with it. Also, validating EDI documents and rejecting bad ones wasn't one of the options (and never mind what the protocols say).  The Golden Rule applies when dealing with automakers: they have the gold, they make the rules.  Sending a rejection document wasn't an option, even when dealing with Ford, which wants an application status document back.

Clay Dowling
Monday, August 11, 2003

Clay - exactly. All the business systems I've had to work with have used a '>', so I simply key off that. Then whether they send a two-digit year or a four-digit I can pick the document apart.

[shrug] it's worked so far. :-)

Philo

Philo
Monday, August 11, 2003

*  Recent Topics

*  Fog Creek Home