About
Welcome to Sandesh Gawande’s website on data warehousing. This website includes wealth of knowledge critical to the role of Enterprise Data Integration Architecture. ETLGuru has been designed for sharing upto date information on Data Integration, ETL and Data Warehousing. Sandesh (Creator of this site) is highly experienced in this field. This portal will share information on ETL, so that Developers, Managers, Architects can have a reliable, usable and impartial source of information.
Why etlguru.com exists? Sandesh started as a Data Architect for OLTP applications. But as he transitioned to Business Intelligence environment, over a period of time he realized that data integration or ETL is seventy to eighty percent of data warehousing effort. Modeling structures are critical but there are many experts to advise on it. But nobody was talking about the dirty work, ETL. In 1999 Sandesh started to take ETL seriously and started to think about strategies that will work for data integration/ETL and data management. The thought process lead him to write a book on ETL. However for many reasons, he realized that he alone cannot finish the ETL book.
In 2005 an ETL book was published by someone else(Kimball group). Not satisfied with the contents, he decided to put his own (real) experience & 200 pages of completed (planned 300 pages) of the book on this website. Thus http://www.etlguru.com/ was born.
We hope the information and services will be useful and valuable to accomplish all your data integration efforts.
Hi Sandesh,
This is a great site. I was browsing to see soe articles about ETL and its prospects and came accross your website, then I thought you would be right person for question I have.
I have been a database developer for the past 12 yrs but I did not get a chance to explore new areas as it was not required for my jobs. But, I feel my skills are outdating and must add new skills to survive in this market. Since, I have lot of interest in Database side, I was thinking adding ETL knowledge would be helpful.
So, I as wondering if you can advice me on how to start learning ETL, Can you suggest some tools, books, training etc., and the approach I should take.
Thanks,
Praveen..
How to extract data from a webpage to any target format using transformations?
Hello Sadesh.
I have encountered a situation.
I want to pull the data from a webpage, using any transformation I have to load the data into target ( Oracle, DB2, Teradata, Flat File or XML File) Any format is fine.
The data will be updated every day on the webpage.
so I have to extract that data using any transformation and load it to any format.
Please suggest me a solution.
Can you send me step by step solution, because I never did such type of ETL. I am new to ETL.
Thank You
Regards
Jai
Hi Sandesh,
I was browsing to see some info about ETL Testing.
The thing is i am having near about 3 years of experience in software testing (manual).
Now i am getting oppotunity to work as ETL tester.
Right now i am evaluating whether it will be suitable for me or not, since i am not having great background of data base testing & dataware house concepts. I want to know what is future scope for ETL tester & what are the opportunities in current market?
while browsing i thought that you would be right person for question I have.
So, if you can advice me on current market requirements for ETL tester & future scope for this job, it will be very helpful for me.
Should i take this as new experience in my testing career or should i stick with application testing(manual/automation) itself.
I know your inputs will really helpful to me in taking proper decision.
thanks in advance.
regards,
Santosh Thorat.
Hi Santosh,
As a QA person it will be good for you to expand your opportunities by learning new things in QA. Data Centric Project testing is not new. Bu only recently it is getting attention. However there are two things that you need to learn before you can become successful in data centric testing.
1. Basic concepts of how to test data
2. Understand the data from POV of busniness
The 2nd part means you not only learn the data warehouse, ETL concepts but also data modeling and business processes. Because this is an emerging area you can soon find yourself in a team leading position.
Sandesh Gawande
http://www.icedq.com
Great site! Hope you’ll begin blogging on expressor Studio at some point.
Michael
I am going through case study where I have to find out suitable technology stack/solution for given situation.
There are 2-3 different multidimensional database systems from where data is retrieved and collected in excel using Essbase Add-in. Also data come from excels.This data is slice and dice in excels and represented in tabular form. These excels can be made part of ppt and published as PDF to users. I want to automate this procedure using ETL tool
The database systems are not MS-SQL Server and can be Oracle or other.
I was planning to create a web application which will have the Devxpress’s ASPxPivot Grid to show data and graphs.
Do you have any recommended architecture for this case study? Any suggestion most welcomed