Use separate db users for deployed components. by cbickel · Pull Request #3876 · apache/openwhisk

cbickel · 2018-07-13T06:44:56Z

With this PR, each deployed component will get it's own database credentials. On doing this, we are able to set the permissions for each component.
E.g. the invoker does not need write access to the subjects- and the whisks db.

The database users and the permission handling is done on wipedb and initdb.

The db-prefix is part of the usernames. This is to avoid clashes if several Openwhisk instances use the same couchdb/cloudant instance.

Related issue and scope

I opened an issue to propose and discuss this change (#????)

My changes affect the following components

Types of changes

Bug fix (generally a non-breaking change which closes an issue).
Enhancement or new feature (adds new functionality).
Breaking change (a bug fix or enhancement which changes existing behavior).

Checklist:

I signed an Apache CLA.
I reviewed the style guides and followed the recommendations (Travis CI will check :).
I added tests to cover my changes.
My changes require further changes to the documentation.
I updated the documentation where necessary.

codecov-io · 2018-07-13T07:22:25Z

Codecov Report

Merging #3876 into master will decrease coverage by 4.76%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3876      +/-   ##
==========================================
- Coverage   75.72%   70.96%   -4.77%     
==========================================
  Files         145      145              
  Lines        6901     6901              
  Branches      417      417              
==========================================
- Hits         5226     4897     -329     
- Misses       1675     2004     +329

Impacted Files	Coverage Δ
...core/database/cosmosdb/RxObservableImplicits.scala	`0% <0%> (-100%)`	⬇️
...core/database/cosmosdb/CosmosDBArtifactStore.scala	`0% <0%> (-95.08%)`	⬇️
...sk/core/database/cosmosdb/CosmosDBViewMapper.scala	`0% <0%> (-92.6%)`	⬇️
...whisk/core/database/cosmosdb/CosmosDBSupport.scala	`0% <0%> (-81.82%)`	⬇️
...abase/cosmosdb/CosmosDBArtifactStoreProvider.scala	`0% <0%> (-58.83%)`	⬇️
...la/whisk/core/database/cosmosdb/CosmosDBUtil.scala	`92% <0%> (-4%)`	⬇️
...rc/main/scala/whisk/common/ForcableSemaphore.scala	`88.46% <0%> (+3.84%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1b8ffcd...0c7ba39. Read the comment docs.

vvraskin · 2018-07-19T08:26:56Z

+    body: |
+      {
+        "cloudant": {
+          {% for item in readerList | union(writerList) | union(adminList) %}"{{ item }}": [ {% if item in readerList %}"_reader"{% if item in writerList %}, "_writer"{% if item in adminList %}, "_admin"{% endif %}{% endif %}{% endif %} ], {% endfor %}


vvraskin · 2018-07-19T08:28:13Z

      "CONFIG_whisk_couchdb_port": "{{ db.port }}"
-      "CONFIG_whisk_couchdb_username": "{{ db.credentials.admin.user }}"
-      "CONFIG_whisk_couchdb_password": "{{ db.credentials.admin.pass }}"
+      "CONFIG_whisk_couchdb_username": "{{ db.credentials.controller.user }}"


should we perhaps use dbUser and dbPass here?
I think the same could be valid for the invoker as well

Please ignore the comment, discussed it in person

vvraskin

LGTM

cbickel · 2018-07-23T09:11:43Z

PG3#2554 🔵

chetanmeh · 2018-07-30T09:15:58Z

I tried to pickup latest master today and for me ansible tasks were failing with below error

$ ansible-playbook -i environments/local properties.yml 

PLAY [ansible] *******************************************************************************************************************************************************************************************************************

TASK [Gathering Facts] ***********************************************************************************************************************************************************************************************************
Monday 30 July 2018  14:29:16 +0530 (0:00:00.276)       0:00:00.276 *********** 
ok: [ansible]

TASK [write whisk.properties template to openwhisk_home] *************************************************************************************************************************************************************************
Monday 30 July 2018  14:29:22 +0530 (0:00:06.372)       0:00:06.649 *********** 
fatal: [ansible]: FAILED! => {"changed": false, "msg": "AnsibleError: An unhandled exception occurred while templating '{u'protocol': u'{{ controllerProtocolForSetup }}', u'extraEnv': u'{{ controller_extraEnv | default({}) }}', u'dir': {u'become': u'{{ controller_dir_become | default(false) }}'}, u'ssl': {u'keystore': {u'path': u'/conf/{{ controllerKeystoreName }}', u'password': u'{{ controllerKeystorePassword }}'}, u'cert': u\"{{ controller_ca_cert | default('controller-openwhisk-server-cert.pem') }}\", u'truststore': {u'path': u'/conf/{{ controllerKeystoreName }}', u'password': u'{{ controllerKeystorePassword }}'}, u'cn': u'openwhisk-controllers', u'key': u\"{{ controller_key | default('controller-openwhisk-server-key.pem') }}\", u'clientAuth': u\"{{ controller_client_auth | default('true') }}\", u'storeFlavor': u'PKCS12'}, u'heap': u\"{{ controller_heap | default('2g') }}\", u'entitlement': {u'spi': u\"{{ controller_entitlement_spi | default('') }}\"}, u'instances': u\"{{ groups['controllers'] | length }}\", u'loglevel': u\"{{ controller_loglevel | default(whisk_loglevel) | default('INFO') }}\", u'timeoutFactor': u'{{ controller_timeout_factor | default(2) }}', u'blackboxFraction': u'{{ controller_blackbox_fraction | default(0.10) }}', u'confdir': u'{{ config_root_dir }}/controller', u'authentication': {u'spi': u\"{{ controller_authentication_spi | default('') }}\"}, u'arguments': u\"{{ controller_arguments | default('') }}\", u'basePort': 10001, u'akka': {u'cluster': {u'bindPort': 2551, u'basePort': 8000, u'host': u\"{{ groups['controllers'] | map('extract', hostvars, 'ansible_host') | list }}\", u'seedNodes': u\"{{ groups['controllers'] | map('extract', hostvars, 'ansible_host') | list }}\"}, u'provider': u'cluster'}, u'loadbalancer': {u'spi': u\"{{ controller_loadbalancer_spi | default('') }}\"}, u'localBookkeeping': u\"{{ controller_local_bookkeeping | default('false') }}\"}'. Error was a <class 'ansible.errors.AnsibleError'>, original message: An unhandled exception occurred while running the lookup plugin 'ini'. Error was a <class 'ConfigParser.NoSectionError'>, original message: No section: u'controller'"}

AnsibleError: An unhandled exception occurred while templating
'{u'protocol': u'{{ controllerProtocolForSetup }}', u'extraEnv': u'{{
controller_extraEnv | default({}) }}', u'dir': {u'become': u'{{
controller_dir_become | default(false) }}'}, u'ssl': {u'keystore': {u'path':
u'/conf/{{ controllerKeystoreName }}', u'password': u'{{
controllerKeystorePassword }}'}, u'cert': u"{{ controller_ca_cert | default
('controller-openwhisk-server-cert.pem') }}", u'truststore': {u'path':
u'/conf/{{ controllerKeystoreName }}', u'password': u'{{
controllerKeystorePassword }}'}, u'cn': u'openwhisk-controllers', u'key': u"{{
controller_key | default('controller-openwhisk-server-key.pem') }}",
u'clientAuth': u"{{ controller_client_auth | default('true') }}",
u'storeFlavor': u'PKCS12'}, u'heap': u"{{ controller_heap | default('2g') }}",
u'entitlement': {u'spi': u"{{ controller_entitlement_spi | default('') }}"},
u'instances': u"{{ groups['controllers'] | length }}", u'loglevel': u"{{
controller_loglevel | default(whisk_loglevel) | default('INFO') }}",
u'timeoutFactor': u'{{ controller_timeout_factor | default(2) }}',
u'blackboxFraction': u'{{ controller_blackbox_fraction | default(0.10) }}',
u'confdir': u'{{ config_root_dir }}/controller', u'authentication': {u'spi':
u"{{ controller_authentication_spi | default('') }}"}, u'arguments': u"{{
controller_arguments | default('') }}", u'basePort': 10001, u'akka':
{u'cluster': {u'bindPort': 2551, u'basePort': 8000, u'host': u"{{
groups['controllers'] | map('extract', hostvars, 'ansible_host') | list }}",
u'seedNodes': u"{{ groups['controllers'] | map('extract', hostvars,
'ansible_host') | list }}"}, u'provider': u'cluster'}, u'loadbalancer': {u'spi':
u"{{ controller_loadbalancer_spi | default('') }}"}, u'localBookkeeping': u"{{
controller_local_bookkeeping | default('false') }}"}'. Error was a <class
'ansible.errors.AnsibleError'>, original message: An unhandled exception
occurred while running the lookup plugin 'ini'. Error was a <class
'ConfigParser.NoSectionError'>, original message: No section: u'controller'

PLAY RECAP ***********************************************************************************************************************************************************************************************************************
ansible                    : ok=1    changed=0    unreachable=0    failed=1

Apparently that was happening because system already had a db_local.ini present which was not getting regenerated and hence was not having newer section for controller. Deleting the existing local and running wskdev couchdb seems to have resolved the problem

cbickel added the wip label Jul 13, 2018

cbickel force-pushed the cdb0 branch from fc6f48a to 558834e Compare July 16, 2018 08:49

cbickel added review Review for this PR has been requested and yet needs to be done. and removed wip labels Jul 16, 2018

cbickel force-pushed the cdb0 branch from 558834e to ba58a02 Compare July 17, 2018 06:04

cbickel requested a review from vvraskin July 18, 2018 10:04

cbickel assigned vvraskin Jul 18, 2018

vvraskin reviewed Jul 19, 2018

View reviewed changes

cbickel force-pushed the cdb0 branch from ba58a02 to b3d1b41 Compare July 19, 2018 10:02

vvraskin approved these changes Jul 20, 2018

View reviewed changes

Use separate credentials for controller and invoker.

0c7ba39

cbickel force-pushed the cdb0 branch from b3d1b41 to 0c7ba39 Compare July 23, 2018 05:56

vvraskin merged commit 02660b4 into apache:master Jul 23, 2018

cbickel deleted the cdb0 branch July 23, 2018 09:51

BillZong pushed a commit to BillZong/openwhisk that referenced this pull request Nov 18, 2019

Use separate credentials for controller and invoker. (apache#3876)

4fc97ba

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use separate db users for deployed components.#3876

Use separate db users for deployed components.#3876
vvraskin merged 1 commit into
apache:masterfrom
cbickel:cdb0

cbickel commented Jul 13, 2018

Uh oh!

codecov-io commented Jul 13, 2018 •

edited

Loading

Uh oh!

vvraskin Jul 19, 2018

Uh oh!

vvraskin Jul 19, 2018

Uh oh!

vvraskin Jul 19, 2018

Uh oh!

vvraskin left a comment

Uh oh!

cbickel commented Jul 23, 2018

Uh oh!

chetanmeh commented Jul 30, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

cbickel commented Jul 13, 2018

Related issue and scope

My changes affect the following components

Types of changes

Checklist:

Uh oh!

codecov-io commented Jul 13, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

vvraskin Jul 19, 2018

Choose a reason for hiding this comment

Uh oh!

vvraskin Jul 19, 2018

Choose a reason for hiding this comment

Uh oh!

vvraskin Jul 19, 2018

Choose a reason for hiding this comment

Uh oh!

vvraskin left a comment

Choose a reason for hiding this comment

Uh oh!

cbickel commented Jul 23, 2018

Uh oh!

chetanmeh commented Jul 30, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov-io commented Jul 13, 2018 •

edited

Loading