Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(k8s): Update pods with correct probes and remove unnecessary dependencies #2362

Merged
merged 7 commits into from
Apr 10, 2021

Conversation

dexter-mh-lee
Copy link
Contributor

List of changes:

  1. Add readiness/liveness probes for all datahub pods (with correct initial delay and period)
  2. Remove file dependencies in main datahub chart (Since all subcharts are under the chart/ directory there is no need to specify file dependencies)
  3. Add redirect paths to frontend ingress (Can be used for ssl-redirects)
  4. Change frontend port to 9002
  5. Add setup jobs for kafka and mysql

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable)

@dexter-mh-lee
Copy link
Contributor Author

Tagging folks who actively use helm charts to make sure the change looks good on your sides. @pedro93 @bernardino @frsann @shakti-garg-saxo

@frsann
Copy link
Contributor

frsann commented Apr 8, 2021

We're rolling our own kustomize-targeted manifests, only loosely based on these charts, so no need to take us into consideration 😅 Thanks for the ping, though!

contrib/kubernetes/datahub/templates/mysql-setup-job.yml Outdated Show resolved Hide resolved
@@ -46,26 +46,38 @@ spec:
imagePullPolicy: {{ .Values.image.pullPolicy }}
ports:
- name: http
containerPort: 9001
containerPort: 9002
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
containerPort: 9002
containerPort: {{ .Values.service.port }}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an internal port. I think it should be hardcoded. service.port field is used in the service.yml to define external port. WDYT?

Comment on lines 55 to 57
initialDelaySeconds: 60
periodSeconds: 30
failureThreshold: 4
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would probably make these come from values

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I make the whole readiness and liveness probe come from values?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i have a suggestion here to only move below properties to values.yaml. The httpGet path can stay hardcoded as this the behavior of app which is not going to change on user wish.

  initialDelaySeconds: 60
  periodSeconds: 30
  failureThreshold: 8

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application.
appVersion: latest #0.3.1
dependencies:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Though these sub-chart dependencies are local, I still believe listing dependencies is good for two reasons:

  1. Good for documentation and chart navigation
  2. one of sub-chart(component) can be toggled off in special deployment scenarios. It is enabled by "condition" property like datahub-mae-consumer.enabled. In our case, we don't need to deploy datahub-ingestion-cron(as we have our own ingestion service), so we just provide parameter "datahub-ingestion.cron.enabled=false"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right. I initially removed this because helm dep update doesn't work for datahub since all the subcharts are in the charts directory. Reverting. Going to keep the version update since multiple setup jobs were added.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated version to 1.1.2 since the change is minor

@@ -4,16 +4,6 @@ A Helm chart for LinkedIn DataHub

Current chart version is `0.1.1`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Version needs to be upgraded as per chart.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Contributor

@shirshanka shirshanka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@shirshanka shirshanka merged commit f293add into datahub-project:master Apr 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants