Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[JUJU-121] Adds support for auto create of instance profiles. #13490

Merged
merged 3 commits into from
Nov 26, 2021

Conversation

tlm
Copy link
Member

@tlm tlm commented Nov 11, 2021

This commit adds support instance profiles auto creation. Including the associated role and policy needed to scope permissions correctly.

Accompanying this PR is also a discourse doc on the permissions Juju requires when talking with AWS.

QA steps

juju bootstrap --bootstrap-constraints="instance-role=auto" aws/ap-southeast-2 test-tlm-controller

You then need to confirm with the aws cli that the associated instance profile elements were created. Make sure that you see a role with the same here attached to the instance profile.

aws iam get-instance-profile --instance-profile-name juju-controller-test-tlm-controller

Add some machines to Juju and deploy a charm with storage to confirm permissions are operating correctly.

Check HA works

$ juju enable-ha
juju enable-ha
maintaining machines: 0
adding machines: 1, 2
$ juju status
Model       Controller     Cloud/Region        Version   SLA          Timestamp
controller  test-tlm-controller aws/ap-southeast-2  2.9.20.1  unsupported  09:34:28+10:00

Machine  State    DNS           Inst id              Series  AZ               Message
0        started  13.211.59.65  i-f00f00f00f00f00f00  focal   ap-southeast-2b  running
1        pending                pending              focal                    attaching aws instance profile arn:aws:iam::123456789:instance-profile/juju-controller-test-tlm-controller
2        pending                pending              focal                    attaching aws instance profile arn:aws:iam::123456789:instance-profile/juju-controller-test-tlm-controller

Check destroy-controller works

$ juju destroy-controller test-tlm-controller --destroy-all-models

Documentation changes

Permission list: https://discourse.charmhub.io/t/juju-aws-permissions/5307
Doc Link: https://discourse.charmhub.io/t/using-aws-instance-profiles-with-juju-2-9/5185/3

Bug reference

N/A

@hpidcock hpidcock changed the title Adds support for auto create of instance profiles. [JUJU-121] Adds support for auto create of instance profiles. Nov 12, 2021
@hpidcock hpidcock added the 2.9 label Nov 12, 2021
provider/ec2/environ.go Outdated Show resolved Hide resolved
provider/ec2/iam.go Outdated Show resolved Hide resolved
provider/ec2/iam.go Outdated Show resolved Hide resolved
provider/ec2/iam_docs.go Outdated Show resolved Hide resolved
provider/ec2/iam_docs.go Outdated Show resolved Hide resolved
@tlm tlm force-pushed the aws-instance-profile branch 3 times, most recently from 25f79c5 to 15e3b32 Compare November 17, 2021 01:51
@tlm tlm marked this pull request as ready for review November 17, 2021 03:30
@tlm tlm force-pushed the aws-instance-profile branch 5 times, most recently from af7337f to 6db3a7b Compare November 18, 2021 02:04
Copy link
Member

@hpidcock hpidcock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only three issues I could see

  • juju enable-ha spins up controllers without an IAM profile.
  • Teardown isn't working
machine-0: 12:58:37 ERROR juju.worker.dependency "undertaker" manifold worker returned unexpected error: cannot destroy cloud resources: destroying storage: listing volumes: operation error EC2: DescribeVolumes, https response error StatusCode: 403, RequestID: eebe4320-5260-448c-91d1-9b74fe7db8ac, api error UnauthorizedOperation: You are not authorized to perform this operation.
  • Instance profile is partially left in bootstrap status (I have a fix for this)

provider/ec2/iam_test.go Outdated Show resolved Hide resolved
tlm and others added 2 commits November 19, 2021 13:32
This commit adds support instance profiles auto creation. Including the associated role and policy needed to scope permissions correctly.
@tlm tlm force-pushed the aws-instance-profile branch 4 times, most recently from 6a90cc8 to ca6be5c Compare November 24, 2021 01:06
Copy link
Member

@hpidcock hpidcock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still testing will approve as soon as I'm done 🏖️

provider/ec2/iam.go Outdated Show resolved Hide resolved
provider/ec2/iam.go Outdated Show resolved Hide resolved
provider/ec2/iam.go Outdated Show resolved Hide resolved
provider/ec2/iam.go Outdated Show resolved Hide resolved
Copy link
Member

@hpidcock hpidcock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Conditional on the cleanup errors are logged.

@hpidcock
Copy link
Member

@tlm lets add a backlog item to deal with instance profile cleanup in github.com/provider/ec2.destroyControllerManagedModels

PR Feedback for Instance Profile work and fixes typo in AWS permission.
@tlm tlm force-pushed the aws-instance-profile branch from ca6be5c to 1a40220 Compare November 26, 2021 00:25
@tlm
Copy link
Member Author

tlm commented Nov 26, 2021

$$merge$$

@jujubot jujubot merged commit 5ba3128 into juju:2.9 Nov 26, 2021
@wallyworld wallyworld mentioned this pull request Nov 26, 2021
jujubot added a commit that referenced this pull request Nov 26, 2021
#13528

Merge 2.9

#13461 Enhanced logging detail when starting side-car agents
#13462 Context queue
#13464 Improve run hook error when charm is missing
#13468 Skip check for microk8s user group on macOS
#13472 Update to latest version of Pebble (exec terminal/interactive params)
#13448 add multi-part yaml support to GetChanges for the bundle facade
#13465 [JUJU-112] manifests and blobs api for registries
#13460 Allow NotifyTarget to return an error
#13482 [JUJU-136] enable juju_machine_lock to display unit machine lock status.
#13477 Added lower bound for run_action wait timeout.
#13490 [JUJU-121] Adds support for auto create of instance profiles.
#13425 [2.9] Expose supported features in show model output
#13426 Remove deprecated juju cli flags
#13422 Provision OpenStack instances with NICs for bindings
#13423 Fix add-machine command help for zones constraint
#13424 Fixed a typo in README.md
#13513 [JUJU-223] Fix panic when retrieving application config
#13517 [JUJU-227] Stop the CAAS storage provisioner worker if the application is dead;
#13519 [JUJU-240] Use the storage prefix when selecting pvcs for sidecar charms
#13520 [JUJU-245] Do not error with permission denied if a controller tries to access removed storage
#13523 Logging for link-layer device updates
#13501 [JUJU-197] Raft lease retries with exceeded dropped
#13514 [JUJU-230] Add additional unit test for WatchAllModels api
#13512 [JUJU-208] Lease Diff
#13515 Remove unused ExportModel method
#13516 [JUJU-235] Remove bogus attachment dead warnings
#13518 [JUJU-237] Fix the failure to update homebrew on none juju/juju
#13498 [JUJU-113] Refactor upgrade controller command
#13502 [JUJU-199] Remove an an old legacy cmr fallback and add upgrade step
#13506 [JUJU-207] Apply synchronously, respond asynchronously
#13508 [JUJU-176] Fix mongo service enable by using start not restart.
#13510 [JUJU-221] Fix vol attachment life asserts for force remove
#13511 [JUJU-215] Ignore notfound error from fetching pod status for updating unit status;
#13500 [JUJU-192] Use yaml.safe_load instead of yaml.load
#13503 [JUJU-202] Lease manager test fix and trace logging
#13504 [JUJU-195] Format logs for mongo version more nicely
#13505 [JUJU-193] [2.9] Populate supported assumes features for k8s provider
#13480 Added Jammy series as supported.
#13488 [JUJU-117] Don't bake defaults when updating config.
#13492 [JUJU-180] Correct affected unit reporting in upgrade-series
#13495 [JUJU-186] Raft client logging levels
#13496 [JUJU-187] Improve the help text for --keep-broken flag
#13493 [JUJU-185] Use the correct ingress resource version for the k8s cluster
#13491 [JUJU-178] Preserve old semantics for run-action concerning wait argument
#13489 [JUJU-177] Check for available disk space before doing a juju backup

```
# Conflicts:
# apiserver/facades/agent/provisioner/provisioninginfo_test.go
# apiserver/facades/client/application/application.go
# apiserver/facades/client/application/application_unit_test.go
# apiserver/facades/client/backups/create.go
# apiserver/facades/client/client/client.go
# apiserver/facades/client/client/client_test.go
# apiserver/facades/client/client/statushistory_test.go
# cmd/juju/action/runaction.go
# cmd/juju/action/runaction_test.go
# cmd/juju/action/waitflag.go
# cmd/juju/commands/upgrademodel.go
# cmd/juju/commands/upgrademodel_test.go
# cmd/jujud/agent/agenttest/agent.go
# go.mod
# go.sum
# mongo/mongo.go
# provider/azure/environ.go
# provider/azure/export_test.go
# provider/azure/storage.go
# provider/azure/storage_test.go
# scripts/win-installer/setup.iss
# snap/snapcraft.yaml
# state/backups/backups.go
# state/backups/backups_test.go
# state/backups/db.go
# state/backups/db_dump_test.go
# state/backups/export_test.go
# state/backups/files_test.go
# version/version.go
```

## QA steps

See PRs

[JUJU-112]: https://warthogs.atlassian.net/browse/JUJU-112?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[JUJU-136]: https://warthogs.atlassian.net/browse/JUJU-136?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[JUJU-121]: https://warthogs.atlassian.net/browse/JUJU-121?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[JUJU-223]: https://warthogs.atlassian.net/browse/JUJU-223?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[JUJU-227]: https://warthogs.atlassian.net/browse/JUJU-227?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants