Tags: getindata/dbt-airflow-factory
Tags
Release 0.35.0 (#105) * Reenable 3.8 (#104) * reenable 3.8 * linting * badge * remove with_test * testing * small fixes * linting * update dbt-graph-builder version * minor version fixed * FIX #105 - Bump version and CHANGELOG for release 0.35.0 --------- Co-authored-by: ptutak-getindata <[email protected]> Co-authored-by: github-actions <[email protected]>
Release 0.32.0 (#94) * chore(docs): remove changelog * chore: prepare version for release * fix: repair versin * Data 369 integrate scheduling airbyte from airflow in the same dag as transformations (#64) * feat: [DATA-369] Add ingestion tasks. * Release 0.30.0 (#72) * Fix missing init * Dependencies fixes (#55) * Fix missing init * Fix test * Fix precommit Co-authored-by: p-pekala <[email protected]> * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TES… (#59) * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TEST_REPORTER_ID is just an ID not credential). Co-authored-by: piotrpekala <[email protected]> * feat: Allows passing config file to the operator (#57) * feat: Allows passing config file to the operator Co-authored-by: bartlomiej.olma.kitopi <[email protected]> * New documentation (#58) * New documentation Co-authored-by: PP <[email protected]> * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TEST_REPORTER_ID is just an ID not credential). * Data 377 (#62) * feat: add config envs if datahub recipe exists * feat: add test to check if env are properly readed * feat: add execution script parameter * feat: add logic behind using execution script * chore: pre-commit * feat: add test for execution script and datahub env * chore: update bash script name * chore: parametrize script path * chore: refactor args builder * chore: refactor tests after changes in args * chore: add defualt value to backward compatible * chore: refactor tests Co-authored-by: Piotr Sierkin <[email protected]> * Data 369 integrate scheduling airbyte from airflow in the same dag as transformations (#64) * feat: [DATA-369] Add ingestion tasks. * Data 412 add save points after each stage of transformations (#61) * feat: [DATA-412] Abstract away task with mapping task from graph to airflow. * Fix/source save point upper (#71) fix: Fixes by builing node depending on the type Co-authored-by: bartlomiej.olma.kitopi <[email protected]> * Additional params k8s pod operator (#69) * feat: add missing params to k8s operator * feat: add secret as optional and add description for new params * chore(docs): add new params * test: add test for new params and move secret from base to qa because of update working in the one direction * chore: remove unsued config because its being inherited from base * chore: reverse testing to check larger area * chore: add required packages * chore: remove required packages * chore: add packages to required * chore: revert changes * chore(docs): fix misspelling * chore: fix misspelling Co-authored-by: Piotr Sierkin <[email protected]> * chore(docs): add changelog * FIX #72 - Bump version and CHANGELOG for release 0.30.0 Co-authored-by: p-pekala <[email protected]> Co-authored-by: github-actions <[email protected]> Co-authored-by: Piotr Pękala <[email protected]> Co-authored-by: bartlomiejolma <[email protected]> Co-authored-by: bartlomiej.olma.kitopi <[email protected]> Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Paweł Kociński <[email protected]> * chore(docs): remove changelog * chore: prepare version for release * fix: repair versin * chore: repair version * chore: repair version number * fix: version * chore(docs): remove changelog * chore: prepare version for release * fix: repair versin * Release 0.30.0 (#72) * Fix missing init * Dependencies fixes (#55) * Fix missing init * Fix test * Fix precommit Co-authored-by: p-pekala <[email protected]> * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TES… (#59) * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TEST_REPORTER_ID is just an ID not credential). Co-authored-by: piotrpekala <[email protected]> * feat: Allows passing config file to the operator (#57) * feat: Allows passing config file to the operator Co-authored-by: bartlomiej.olma.kitopi <[email protected]> * New documentation (#58) * New documentation Co-authored-by: PP <[email protected]> * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TEST_REPORTER_ID is just an ID not credential). * Data 377 (#62) * feat: add config envs if datahub recipe exists * feat: add test to check if env are properly readed * feat: add execution script parameter * feat: add logic behind using execution script * chore: pre-commit * feat: add test for execution script and datahub env * chore: update bash script name * chore: parametrize script path * chore: refactor args builder * chore: refactor tests after changes in args * chore: add defualt value to backward compatible * chore: refactor tests Co-authored-by: Piotr Sierkin <[email protected]> * Data 369 integrate scheduling airbyte from airflow in the same dag as transformations (#64) * feat: [DATA-369] Add ingestion tasks. * Data 412 add save points after each stage of transformations (#61) * feat: [DATA-412] Abstract away task with mapping task from graph to airflow. * Fix/source save point upper (#71) fix: Fixes by builing node depending on the type Co-authored-by: bartlomiej.olma.kitopi <[email protected]> * Additional params k8s pod operator (#69) * feat: add missing params to k8s operator * feat: add secret as optional and add description for new params * chore(docs): add new params * test: add test for new params and move secret from base to qa because of update working in the one direction * chore: remove unsued config because its being inherited from base * chore: reverse testing to check larger area * chore: add required packages * chore: remove required packages * chore: add packages to required * chore: revert changes * chore(docs): fix misspelling * chore: fix misspelling Co-authored-by: Piotr Sierkin <[email protected]> * chore(docs): add changelog * FIX #72 - Bump version and CHANGELOG for release 0.30.0 Co-authored-by: p-pekala <[email protected]> Co-authored-by: github-actions <[email protected]> Co-authored-by: Piotr Pękala <[email protected]> Co-authored-by: bartlomiejolma <[email protected]> Co-authored-by: bartlomiej.olma.kitopi <[email protected]> Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Paweł Kociński <[email protected]> * chore(docs): remove changelog * chore: prepare version for release * fix: repair versin * chore: repair version * chore: repair version number * fix: version * Feat/pod timeout param (#82) * feat: exposes startup timeout seconds for pod operator * Bash operator cherry pick (#83) * Bash operator * Fix/source dependency without meta (#84) * Fix missing init * Dependencies fixes (#55) * Fix missing init * Fix test * Fix precommit Co-authored-by: p-pekala <[email protected]> * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TES… (#59) * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TEST_REPORTER_ID is just an ID not credential). Co-authored-by: piotrpekala <[email protected]> * feat: Allows passing config file to the operator (#57) * feat: Allows passing config file to the operator Co-authored-by: bartlomiej.olma.kitopi <[email protected]> * New documentation (#58) * New documentation Co-authored-by: PP <[email protected]> * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TEST_REPORTER_ID is just an ID not credential). * Data 377 (#62) * feat: add config envs if datahub recipe exists * feat: add test to check if env are properly readed * feat: add execution script parameter * feat: add logic behind using execution script * chore: pre-commit * feat: add test for execution script and datahub env * chore: update bash script name * chore: parametrize script path * chore: refactor args builder * chore: refactor tests after changes in args * chore: add defualt value to backward compatible * chore: refactor tests Co-authored-by: Piotr Sierkin <[email protected]> * Data 369 integrate scheduling airbyte from airflow in the same dag as transformations (#64) * feat: [DATA-369] Add ingestion tasks. * Data 412 add save points after each stage of transformations (#61) * feat: [DATA-412] Abstract away task with mapping task from graph to airflow. * fix: adds test that confirms issue * fix: checks if node is there before adding edge with it * fix: name change * fix: test formatting --------- Co-authored-by: p-pekala <[email protected]> Co-authored-by: github-actions <[email protected]> Co-authored-by: Piotr Pękala <[email protected]> Co-authored-by: bartlomiej.olma.kitopi <[email protected]> Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Paweł Kociński <[email protected]> * chore: bump isort * Do not use deprecated Kubernetes settings (#87) * Do not use deprecated Kubernetes settings Since Airflow 2.3, KubernetesPodOperator should not use "node_selectors" and "resources" parameters, but "node_selector" and "container_resources" respectively instead. * Fix `test_notifications` * Fix `test_configuration` * Introduce `IS_FIRST_AIRFLOW_VERSION` constant * Fix CHANGELOG for release 0.31.0 * fix: replace default param with None (#89) * fix: replace default param with None * chore(docs): add description to config_file * chore(docs): update changelog * fix: add config file for readthedocs due to urlib3 release * chore: remove comments * chore: update param docs --------- Co-authored-by: Piotr Sierkin <[email protected]> * Replace creating dag with dbt graph builder (#92) * update dependencies * replace creating dag with dbt-graph-builder * version bump * python 3.10 * python 3.10 * pre-commit linting * python version badge update * review changes * linting * python 3.8 minimum * python 3.8 * badge update * reverse version change (#93) * FIX #94 - Bump version and CHANGELOG for release 0.32.0 --------- Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Paweł Kociński <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: p-pekala <[email protected]> Co-authored-by: github-actions <[email protected]> Co-authored-by: Piotr Pękala <[email protected]> Co-authored-by: bartlomiejolma <[email protected]> Co-authored-by: bartlomiej.olma.kitopi <[email protected]> Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Maciej Bryński <[email protected]> Co-authored-by: Andrzej Swatowski <[email protected]> Co-authored-by: Andrzej Swatowski <[email protected]> Co-authored-by: ptutak-getindata <[email protected]>
Release 0.31.1 (#91) * chore(docs): remove changelog * chore: prepare version for release * fix: repair versin * Data 369 integrate scheduling airbyte from airflow in the same dag as transformations (#64) * feat: [DATA-369] Add ingestion tasks. * Release 0.30.0 (#72) * Fix missing init * Dependencies fixes (#55) * Fix missing init * Fix test * Fix precommit Co-authored-by: p-pekala <[email protected]> * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TES… (#59) * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TEST_REPORTER_ID is just an ID not credential). Co-authored-by: piotrpekala <[email protected]> * feat: Allows passing config file to the operator (#57) * feat: Allows passing config file to the operator Co-authored-by: bartlomiej.olma.kitopi <[email protected]> * New documentation (#58) * New documentation Co-authored-by: PP <[email protected]> * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TEST_REPORTER_ID is just an ID not credential). * Data 377 (#62) * feat: add config envs if datahub recipe exists * feat: add test to check if env are properly readed * feat: add execution script parameter * feat: add logic behind using execution script * chore: pre-commit * feat: add test for execution script and datahub env * chore: update bash script name * chore: parametrize script path * chore: refactor args builder * chore: refactor tests after changes in args * chore: add defualt value to backward compatible * chore: refactor tests Co-authored-by: Piotr Sierkin <[email protected]> * Data 369 integrate scheduling airbyte from airflow in the same dag as transformations (#64) * feat: [DATA-369] Add ingestion tasks. * Data 412 add save points after each stage of transformations (#61) * feat: [DATA-412] Abstract away task with mapping task from graph to airflow. * Fix/source save point upper (#71) fix: Fixes by builing node depending on the type Co-authored-by: bartlomiej.olma.kitopi <[email protected]> * Additional params k8s pod operator (#69) * feat: add missing params to k8s operator * feat: add secret as optional and add description for new params * chore(docs): add new params * test: add test for new params and move secret from base to qa because of update working in the one direction * chore: remove unsued config because its being inherited from base * chore: reverse testing to check larger area * chore: add required packages * chore: remove required packages * chore: add packages to required * chore: revert changes * chore(docs): fix misspelling * chore: fix misspelling Co-authored-by: Piotr Sierkin <[email protected]> * chore(docs): add changelog * FIX #72 - Bump version and CHANGELOG for release 0.30.0 Co-authored-by: p-pekala <[email protected]> Co-authored-by: github-actions <[email protected]> Co-authored-by: Piotr Pękala <[email protected]> Co-authored-by: bartlomiejolma <[email protected]> Co-authored-by: bartlomiej.olma.kitopi <[email protected]> Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Paweł Kociński <[email protected]> * chore(docs): remove changelog * chore: prepare version for release * fix: repair versin * chore: repair version * chore: repair version number * fix: version * chore(docs): remove changelog * chore: prepare version for release * fix: repair versin * Release 0.30.0 (#72) * Fix missing init * Dependencies fixes (#55) * Fix missing init * Fix test * Fix precommit Co-authored-by: p-pekala <[email protected]> * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TES… (#59) * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TEST_REPORTER_ID is just an ID not credential). Co-authored-by: piotrpekala <[email protected]> * feat: Allows passing config file to the operator (#57) * feat: Allows passing config file to the operator Co-authored-by: bartlomiej.olma.kitopi <[email protected]> * New documentation (#58) * New documentation Co-authored-by: PP <[email protected]> * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TEST_REPORTER_ID is just an ID not credential). * Data 377 (#62) * feat: add config envs if datahub recipe exists * feat: add test to check if env are properly readed * feat: add execution script parameter * feat: add logic behind using execution script * chore: pre-commit * feat: add test for execution script and datahub env * chore: update bash script name * chore: parametrize script path * chore: refactor args builder * chore: refactor tests after changes in args * chore: add defualt value to backward compatible * chore: refactor tests Co-authored-by: Piotr Sierkin <[email protected]> * Data 369 integrate scheduling airbyte from airflow in the same dag as transformations (#64) * feat: [DATA-369] Add ingestion tasks. * Data 412 add save points after each stage of transformations (#61) * feat: [DATA-412] Abstract away task with mapping task from graph to airflow. * Fix/source save point upper (#71) fix: Fixes by builing node depending on the type Co-authored-by: bartlomiej.olma.kitopi <[email protected]> * Additional params k8s pod operator (#69) * feat: add missing params to k8s operator * feat: add secret as optional and add description for new params * chore(docs): add new params * test: add test for new params and move secret from base to qa because of update working in the one direction * chore: remove unsued config because its being inherited from base * chore: reverse testing to check larger area * chore: add required packages * chore: remove required packages * chore: add packages to required * chore: revert changes * chore(docs): fix misspelling * chore: fix misspelling Co-authored-by: Piotr Sierkin <[email protected]> * chore(docs): add changelog * FIX #72 - Bump version and CHANGELOG for release 0.30.0 Co-authored-by: p-pekala <[email protected]> Co-authored-by: github-actions <[email protected]> Co-authored-by: Piotr Pękala <[email protected]> Co-authored-by: bartlomiejolma <[email protected]> Co-authored-by: bartlomiej.olma.kitopi <[email protected]> Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Paweł Kociński <[email protected]> * chore(docs): remove changelog * chore: prepare version for release * fix: repair versin * chore: repair version * chore: repair version number * fix: version * Feat/pod timeout param (#82) * feat: exposes startup timeout seconds for pod operator * Bash operator cherry pick (#83) * Bash operator * Fix/source dependency without meta (#84) * Fix missing init * Dependencies fixes (#55) * Fix missing init * Fix test * Fix precommit Co-authored-by: p-pekala <[email protected]> * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TES… (#59) * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TEST_REPORTER_ID is just an ID not credential). Co-authored-by: piotrpekala <[email protected]> * feat: Allows passing config file to the operator (#57) * feat: Allows passing config file to the operator Co-authored-by: bartlomiej.olma.kitopi <[email protected]> * New documentation (#58) * New documentation Co-authored-by: PP <[email protected]> * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TEST_REPORTER_ID is just an ID not credential). * Data 377 (#62) * feat: add config envs if datahub recipe exists * feat: add test to check if env are properly readed * feat: add execution script parameter * feat: add logic behind using execution script * chore: pre-commit * feat: add test for execution script and datahub env * chore: update bash script name * chore: parametrize script path * chore: refactor args builder * chore: refactor tests after changes in args * chore: add defualt value to backward compatible * chore: refactor tests Co-authored-by: Piotr Sierkin <[email protected]> * Data 369 integrate scheduling airbyte from airflow in the same dag as transformations (#64) * feat: [DATA-369] Add ingestion tasks. * Data 412 add save points after each stage of transformations (#61) * feat: [DATA-412] Abstract away task with mapping task from graph to airflow. * fix: adds test that confirms issue * fix: checks if node is there before adding edge with it * fix: name change * fix: test formatting --------- Co-authored-by: p-pekala <[email protected]> Co-authored-by: github-actions <[email protected]> Co-authored-by: Piotr Pękala <[email protected]> Co-authored-by: bartlomiej.olma.kitopi <[email protected]> Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Paweł Kociński <[email protected]> * chore: bump isort * Do not use deprecated Kubernetes settings (#87) * Do not use deprecated Kubernetes settings Since Airflow 2.3, KubernetesPodOperator should not use "node_selectors" and "resources" parameters, but "node_selector" and "container_resources" respectively instead. * Fix `test_notifications` * Fix `test_configuration` * Introduce `IS_FIRST_AIRFLOW_VERSION` constant * Fix CHANGELOG for release 0.31.0 * fix: replace default param with None (#89) * fix: replace default param with None * chore(docs): add description to config_file * chore(docs): update changelog * fix: add config file for readthedocs due to urlib3 release * chore: remove comments * chore: update param docs --------- Co-authored-by: Piotr Sierkin <[email protected]> * FIX #91 - Bump version and CHANGELOG for release 0.31.1 --------- Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Paweł Kociński <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: p-pekala <[email protected]> Co-authored-by: github-actions <[email protected]> Co-authored-by: Piotr Pękala <[email protected]> Co-authored-by: bartlomiejolma <[email protected]> Co-authored-by: bartlomiej.olma.kitopi <[email protected]> Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Maciej Bryński <[email protected]> Co-authored-by: Andrzej Swatowski <[email protected]> Co-authored-by: Andrzej Swatowski <[email protected]>
Release 0.31.0 (#88) * Do not use deprecated Kubernetes settings (#87) * Do not use deprecated Kubernetes settings Since Airflow 2.3, KubernetesPodOperator should not use "node_selectors" and "resources" parameters, but "node_selector" and "container_resources" respectively instead. * Fix `test_notifications` * Fix `test_configuration` * Introduce `IS_FIRST_AIRFLOW_VERSION` constant * FIX #88 - Bump version and CHANGELOG for release 0.31.0
Release 0.30.0 (#85) * chore(docs): remove changelog * chore: prepare version for release * fix: repair versin * Data 369 integrate scheduling airbyte from airflow in the same dag as transformations (#64) * feat: [DATA-369] Add ingestion tasks. * Release 0.30.0 (#72) * Fix missing init * Dependencies fixes (#55) * Fix missing init * Fix test * Fix precommit Co-authored-by: p-pekala <[email protected]> * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TES… (#59) * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TEST_REPORTER_ID is just an ID not credential). Co-authored-by: piotrpekala <[email protected]> * feat: Allows passing config file to the operator (#57) * feat: Allows passing config file to the operator Co-authored-by: bartlomiej.olma.kitopi <[email protected]> * New documentation (#58) * New documentation Co-authored-by: PP <[email protected]> * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TEST_REPORTER_ID is just an ID not credential). * Data 377 (#62) * feat: add config envs if datahub recipe exists * feat: add test to check if env are properly readed * feat: add execution script parameter * feat: add logic behind using execution script * chore: pre-commit * feat: add test for execution script and datahub env * chore: update bash script name * chore: parametrize script path * chore: refactor args builder * chore: refactor tests after changes in args * chore: add defualt value to backward compatible * chore: refactor tests Co-authored-by: Piotr Sierkin <[email protected]> * Data 369 integrate scheduling airbyte from airflow in the same dag as transformations (#64) * feat: [DATA-369] Add ingestion tasks. * Data 412 add save points after each stage of transformations (#61) * feat: [DATA-412] Abstract away task with mapping task from graph to airflow. * Fix/source save point upper (#71) fix: Fixes by builing node depending on the type Co-authored-by: bartlomiej.olma.kitopi <[email protected]> * Additional params k8s pod operator (#69) * feat: add missing params to k8s operator * feat: add secret as optional and add description for new params * chore(docs): add new params * test: add test for new params and move secret from base to qa because of update working in the one direction * chore: remove unsued config because its being inherited from base * chore: reverse testing to check larger area * chore: add required packages * chore: remove required packages * chore: add packages to required * chore: revert changes * chore(docs): fix misspelling * chore: fix misspelling Co-authored-by: Piotr Sierkin <[email protected]> * chore(docs): add changelog * FIX #72 - Bump version and CHANGELOG for release 0.30.0 Co-authored-by: p-pekala <[email protected]> Co-authored-by: github-actions <[email protected]> Co-authored-by: Piotr Pękala <[email protected]> Co-authored-by: bartlomiejolma <[email protected]> Co-authored-by: bartlomiej.olma.kitopi <[email protected]> Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Paweł Kociński <[email protected]> * chore(docs): remove changelog * chore: prepare version for release * fix: repair versin * chore: repair version * chore: repair version number * fix: version * chore(docs): remove changelog * chore: prepare version for release * fix: repair versin * Release 0.30.0 (#72) * Fix missing init * Dependencies fixes (#55) * Fix missing init * Fix test * Fix precommit Co-authored-by: p-pekala <[email protected]> * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TES… (#59) * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TEST_REPORTER_ID is just an ID not credential). Co-authored-by: piotrpekala <[email protected]> * feat: Allows passing config file to the operator (#57) * feat: Allows passing config file to the operator Co-authored-by: bartlomiej.olma.kitopi <[email protected]> * New documentation (#58) * New documentation Co-authored-by: PP <[email protected]> * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TEST_REPORTER_ID is just an ID not credential). * Data 377 (#62) * feat: add config envs if datahub recipe exists * feat: add test to check if env are properly readed * feat: add execution script parameter * feat: add logic behind using execution script * chore: pre-commit * feat: add test for execution script and datahub env * chore: update bash script name * chore: parametrize script path * chore: refactor args builder * chore: refactor tests after changes in args * chore: add defualt value to backward compatible * chore: refactor tests Co-authored-by: Piotr Sierkin <[email protected]> * Data 369 integrate scheduling airbyte from airflow in the same dag as transformations (#64) * feat: [DATA-369] Add ingestion tasks. * Data 412 add save points after each stage of transformations (#61) * feat: [DATA-412] Abstract away task with mapping task from graph to airflow. * Fix/source save point upper (#71) fix: Fixes by builing node depending on the type Co-authored-by: bartlomiej.olma.kitopi <[email protected]> * Additional params k8s pod operator (#69) * feat: add missing params to k8s operator * feat: add secret as optional and add description for new params * chore(docs): add new params * test: add test for new params and move secret from base to qa because of update working in the one direction * chore: remove unsued config because its being inherited from base * chore: reverse testing to check larger area * chore: add required packages * chore: remove required packages * chore: add packages to required * chore: revert changes * chore(docs): fix misspelling * chore: fix misspelling Co-authored-by: Piotr Sierkin <[email protected]> * chore(docs): add changelog * FIX #72 - Bump version and CHANGELOG for release 0.30.0 Co-authored-by: p-pekala <[email protected]> Co-authored-by: github-actions <[email protected]> Co-authored-by: Piotr Pękala <[email protected]> Co-authored-by: bartlomiejolma <[email protected]> Co-authored-by: bartlomiej.olma.kitopi <[email protected]> Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Paweł Kociński <[email protected]> * chore(docs): remove changelog * chore: prepare version for release * fix: repair versin * chore: repair version * chore: repair version number * fix: version * Feat/pod timeout param (#82) * feat: exposes startup timeout seconds for pod operator * Bash operator cherry pick (#83) * Bash operator * Fix/source dependency without meta (#84) * Fix missing init * Dependencies fixes (#55) * Fix missing init * Fix test * Fix precommit Co-authored-by: p-pekala <[email protected]> * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TES… (#59) * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TEST_REPORTER_ID is just an ID not credential). Co-authored-by: piotrpekala <[email protected]> * feat: Allows passing config file to the operator (#57) * feat: Allows passing config file to the operator Co-authored-by: bartlomiej.olma.kitopi <[email protected]> * New documentation (#58) * New documentation Co-authored-by: PP <[email protected]> * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TEST_REPORTER_ID is just an ID not credential). * Data 377 (#62) * feat: add config envs if datahub recipe exists * feat: add test to check if env are properly readed * feat: add execution script parameter * feat: add logic behind using execution script * chore: pre-commit * feat: add test for execution script and datahub env * chore: update bash script name * chore: parametrize script path * chore: refactor args builder * chore: refactor tests after changes in args * chore: add defualt value to backward compatible * chore: refactor tests Co-authored-by: Piotr Sierkin <[email protected]> * Data 369 integrate scheduling airbyte from airflow in the same dag as transformations (#64) * feat: [DATA-369] Add ingestion tasks. * Data 412 add save points after each stage of transformations (#61) * feat: [DATA-412] Abstract away task with mapping task from graph to airflow. * fix: adds test that confirms issue * fix: checks if node is there before adding edge with it * fix: name change * fix: test formatting --------- Co-authored-by: p-pekala <[email protected]> Co-authored-by: github-actions <[email protected]> Co-authored-by: Piotr Pękala <[email protected]> Co-authored-by: bartlomiej.olma.kitopi <[email protected]> Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Paweł Kociński <[email protected]> * chore: bump isort * FIX #85 - Bump version and CHANGELOG for release 0.30.0 --------- Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Paweł Kociński <[email protected]> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: p-pekala <[email protected]> Co-authored-by: github-actions <[email protected]> Co-authored-by: Piotr Pękala <[email protected]> Co-authored-by: bartlomiejolma <[email protected]> Co-authored-by: bartlomiej.olma.kitopi <[email protected]> Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Maciej Bryński <[email protected]>
Release 0.29.0 (#65) * Fix missing init * Dependencies fixes (#55) * Fix missing init * Fix test * Fix precommit Co-authored-by: p-pekala <[email protected]> * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TES… (#59) * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TEST_REPORTER_ID is just an ID not credential). Co-authored-by: piotrpekala <[email protected]> * feat: Allows passing config file to the operator (#57) * feat: Allows passing config file to the operator Co-authored-by: bartlomiej.olma.kitopi <[email protected]> * New documentation (#58) * New documentation Co-authored-by: PP <[email protected]> * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TEST_REPORTER_ID is just an ID not credential). * Data 377 (#62) * feat: add config envs if datahub recipe exists * feat: add test to check if env are properly readed * feat: add execution script parameter * feat: add logic behind using execution script * chore: pre-commit * feat: add test for execution script and datahub env * chore: update bash script name * chore: parametrize script path * chore: refactor args builder * chore: refactor tests after changes in args * chore: add defualt value to backward compatible * chore: refactor tests Co-authored-by: Piotr Sierkin <[email protected]> * Data 369 integrate scheduling airbyte from airflow in the same dag as transformations (#64) * feat: [DATA-369] Add ingestion tasks. * Data 412 add save points after each stage of transformations (#61) * feat: [DATA-412] Abstract away task with mapping task from graph to airflow. * FIX #65 - Bump version and CHANGELOG for release 0.29.0 Co-authored-by: p-pekala <[email protected]> Co-authored-by: github-actions <[email protected]> Co-authored-by: Piotr Pękala <[email protected]> Co-authored-by: bartlomiejolma <[email protected]> Co-authored-by: bartlomiej.olma.kitopi <[email protected]> Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Paweł Kociński <[email protected]>
Release 0.28.0 (#63) * Fix missing init * Dependencies fixes (#55) * Fix missing init * Fix test * Fix precommit Co-authored-by: p-pekala <[email protected]> * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TES… (#59) * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TEST_REPORTER_ID is just an ID not credential). Co-authored-by: piotrpekala <[email protected]> * feat: Allows passing config file to the operator (#57) * feat: Allows passing config file to the operator Co-authored-by: bartlomiej.olma.kitopi <[email protected]> * New documentation (#58) * New documentation Co-authored-by: PP <[email protected]> * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TEST_REPORTER_ID is just an ID not credential). * Data 377 (#62) * feat: add config envs if datahub recipe exists * feat: add test to check if env are properly readed * feat: add execution script parameter * feat: add logic behind using execution script * chore: pre-commit * feat: add test for execution script and datahub env * chore: update bash script name * chore: parametrize script path * chore: refactor args builder * chore: refactor tests after changes in args * chore: add defualt value to backward compatible * chore: refactor tests Co-authored-by: Piotr Sierkin <[email protected]> * FIX #63 - Bump version and CHANGELOG for release 0.28.0 Co-authored-by: p-pekala <[email protected]> Co-authored-by: github-actions <[email protected]> Co-authored-by: Piotr Pękala <[email protected]> Co-authored-by: bartlomiejolma <[email protected]> Co-authored-by: bartlomiej.olma.kitopi <[email protected]> Co-authored-by: Piotr Sierkin <[email protected]> Co-authored-by: Piotr Sierkin <[email protected]>
Release 0.27.0 (#60) * Moving CC_TEST_REPORTER_ID from secrets to allow contribution (CC_TEST_REPORTER_ID is just an ID not credential). Co-authored-by: piotrpekala <[email protected]> * feat: Allows passing config file to the operator (#57) Co-authored-by: bartlomiej.olma.kitopi <[email protected]> * New documentation Co-authored-by: PP <[email protected]>
PreviousNext