Fog Creek Software
Discussion Board




Extraction Translation Load

Extraction Translation Load or ETL is a buzzword my company is starting to notice. The company I am at works with lots of major clients and we are often given large datasets and have to give back large datasets in all sorts of non-standard formats. We have had to use SQL Server 7.0/2000 DTS and home grown loaders (ASP or VB) and we are wondering if you guys have had some success with ETL tools on the market?

Writing all these translators is taking too much time for our company and often can't be justified. But not many of our clients have the funding to move to standards like XML and corporate-standard schemas, although some have picked up standard ERP systems which supports standard transport methods out of the box. So we end up writing a lot of the loaders and extractors at the end.

We are wondering which 3rd party ETL tool is good at parsing out strange formats? Which one is reliable? Which ones ties tightly with major DB engines for fast loading? Or are you guys sticking with the loaders and translators that comes from the same vendor as your DB engine (SQL Server DTS Editor comes to mind)?

Any war stories or useful notes would be appreciated. Also if any of you can suggest a good low cost 3rd party ETL tool on the market that would be much appreciated.

Many many thanks, much appreciated!

Li-fan Chen
Friday, January 10, 2003

You might want to look at Microsoft Biztalk.  Think it does what you're looking for, though much of the work you'll still need to do.  I haven't done more than install/mess around with it so you'll want to check it out for yourself...

GiorgioG
Friday, January 10, 2003

I know this is probably not immediately helpful but it deserves a little consideration.

You speak as if you already have one or several translators written and functional. If this is the case, have you guys considered making a business doing this? You need it and can't find it: if other people need it, that's a good market.

And you guys have the expert knowledge to pull it off (having done it multiple times already).

Just a thought and a half...

Mike Swieton
Friday, January 10, 2003

Data warehousing is one area that uses these tools in depth, sometimes layered upon one another.

Check out sites like the following:

http://www.dwinfocenter.org/clean.html

and similar sites.

Wolf Bogacz
Friday, January 10, 2003

http://www.datajunction.com/

Scot
Saturday, January 11, 2003

Just want to thank you guys for all the great ideas. I'll look into each of them and see where they lead!

Li-fan Chen
Saturday, January 11, 2003

*  Recent Topics

*  Fog Creek Home