Final project from Coursera course "Python Project for Data Engineering"

For this project, I will assume the role of data engineer working for an international financial analysis company. The company tracks stock prices, commodities, forex rates, inflation rates. The job is to extract financial data from various sources like websites, APIs and files provided by various financial analysis firms. After I collect the data, I extract the data of interest to the company and transform it based on the requirements given. Once the transformation is complete I load that data into the database.

• Collect data using APIs

• Collect data using webscraping.

• Download files to process.

• Read csv, xml and json file types.

• Extract data from the above file types.

• Transform data.

• Use the built in logging module.

• Save the transformed data in a ready-to-load format which data engineers can use to load the data.