Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize the code framework #79

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Conversation

garenchan
Copy link

@garenchan garenchan commented Aug 16, 2022

What problem are we solving?

When we use the s3fs mounter, even if multiple pods use the same volume, the driver will start a process for each of them to mount the volume. This is wasteful of resources and bad for management.

o ps -ef | grep s3fs | grep pod
root     26654  3594  0 15:47 ?        00:00:00 s3fs custom / /var/lib/kubelet/pods/7e72ff6c-8096-4de3-973f-9d1a97fc3a89/volumes/kubernetes.io~csi/pvc-1a2313d1-ae74-45ee-bfa4-63106d7084ec/mount -o use_path_request_style -o url=http://10.134.80.223:9000 -o endpoint= -o allow_other -o mp_umask=000
root     34834  3594  0 19:32 ?        00:00:00 s3fs custom / /var/lib/kubelet/pods/5b8a4cb7-3897-42eb-94cf-049d0485a11b/volumes/kubernetes.io~csi/pvc-1a2313d1-ae74-45ee-bfa4-63106d7084ec/mount -o use_path_request_style -o url=http://10.134.80.223:9000 -o endpoint= -o allow_other -o mp_umask=000

How are we solving the problem?

The idea is that pods using the same volume need to share a mount process.
We currently solve this problem based on the CSI specification.

  1. When a volume is first used on a node, NodeStageVolume will start a process to mount it to host's /var/lib/kubelet/plugins/kubernetes.io/csi/pv/<pvc>/globalmount directory.
  2. Later when other pod uses the volume, NodePublishVolume just needs to bind mount /var/lib/kubelet/plugins/kubernetes.io/csi/pv/<pvc>/globalmount to /var/lib/kubelet/pods/<pod>/volumes/kubernetes.io~csi/<pvc>/mount directory.
  3. If the pod using the volume on the node is deleted, then NodeUnpublishVolume only needs to unmount /var/lib/kubelet/pods/<pod>/volumes/kubernetes.io~csi/<pvc>/mount.
  4. Finally, when the volume is not used by any pods on the node, NodeUnstageVolume will unmount the volume from /var/lib/kubelet/plugins/kubernetes.io/csi/pv/<pvc>/globalmount directory.

We made a significant change to the code to make it easier to perform similar optimizations for other mounters.

Additionally, according to the specification, we also consider scenarios for handling concurrent calls.
https://github.com/container-storage-interface/spec/blob/master/spec.md#concurrency

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant