This is a small Dataflow Job that receives a message via pubsub every time someone accesses a shortened URL. It accumulates the items using a Fixed Time Window, groups by Id and updates FireStore with the amount of clicks.
The technology behind it:
In the terminal run the following command:
python -m venv url-shortener-counter
source tutorial-env/bin/activate
pip install -r requirements.txtIn the terminal run the following command:
conda env create --file=environment.yml
source activate url-shortener-counterSet environment variables:
export PROJECT_ID=[your_projec_id]
export GOOGLE_APPLICATION_CREDENTIALS=/temp/url-counter-handy-service-account.jsonIn the terminal run the following command:
python main.py \
--project_id=${PROJECT_ID} \
--input_subscription=projects/${PROJECT_ID}/subscriptions/url-clicks-counter \
--collection=urls \
--fixed_windows=60Template generation:
python main.py \
--runner=DataflowRunner \
--project=${PROJECT_ID} \
--staging_location=gs://${GCP_BUCKET}/staging \
--temp_location=gs://${GCP_BUCKET}/temp \
--template_location=gs://${GCP_BUCKET}/templates/url-shortener-counter-1.0.0 \
--region=southamerica-east1 \
--project_id=${PROJECT_ID} \
--input_subscription=projects/${PROJECT_ID}/subscriptions/url-clicks-counter \
--collection=urls \
--fixed_windows=60 \
--requirements_file=requirements-dataflow.txtRun the template on Dataflow:
See the documentation Running classic templates
