Project Description:
- This year, inflation has affected the prices of goods and services in many countries. same as Thailand.
- Due to personal interest and is one of the people affected by the problem. Makes me want to know how the price of each type of product has changed But found that most of the information in Thailand is not in a format that can be tracked and is easily accessible to the general public.
What the problem the project solves ?
- Data Warehouse of product prices in Thailand for use in data analysis, dashboard, APIs, etc.
- A dashboard for easy access and monitoring of product prices.
- A public dashboard that can easily monitor and interpret a data visualization.
-
Infrastructure as code (Iac Tools)
- Terraform
-
Data Lake
- Google Cloud Storage
-
Data Warehouse
- Big Query
-
Step
- Get data product price from from data.mog.co.th (Ministry of Comerce's Public Data)
- Load source data to data lake (GCS)
- Daily update data by Apache Airflow
-
Pipeline
- Use Apache Airflow for data pipeline.
-
Seeds Datasets
- Data from 2010 - 2021 will use seeding data concept.
- Product Price data will start daily update from 2022.
- Use Google Big Query for Data Warehouse
- Use dbt cloud for transform data
Thailand Agricultural Price Dashboard
