Skip to content

blk/kernel : Make bdev stop immediately #60791

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 13, 2025

Conversation

Jayaprakash-ibm
Copy link
Contributor

@Jayaprakash-ibm Jayaprakash-ibm commented Nov 21, 2024

This change addresses the 200ms delay observed in the _aio_stop function during the completion of bdev operations.

The _aio_stop function was modified to introduce a wakeup mechanism using a dummy operation

Fixes : #68881

Contribution Guidelines

  • To sign and title your commits, please refer to Submitting Patches to Ceph.

  • If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

  • When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an x between the brackets: [x]. Spaces and capitalization matter when checking off items this way.

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows
  • jenkins test rook e2e

@Jayaprakash-ibm Jayaprakash-ibm requested a review from a team as a code owner November 21, 2024 15:20
@github-actions github-actions bot added the core label Nov 21, 2024
@aclamk aclamk changed the title blk/kernel : Reduce bdev Completion Delay in _aio_stop blk/kernel : Make bdev stop immediately Nov 21, 2024
@Jayaprakash-ibm Jayaprakash-ibm force-pushed the jaya-branch branch 4 times, most recently from 3d3b0ca to ce74e3c Compare November 21, 2024 19:27
@Jayaprakash-ibm Jayaprakash-ibm requested a review from a team as a code owner November 21, 2024 19:27
@Jayaprakash-ibm Jayaprakash-ibm requested review from dnyanee1997 and Achintk1491 and removed request for a team November 21, 2024 19:27
@Jayaprakash-ibm Jayaprakash-ibm force-pushed the jaya-branch branch 5 times, most recently from e88cc7c to 0104c4b Compare November 21, 2024 19:48
@aclamk
Copy link
Contributor

aclamk commented Nov 22, 2024

jenkins test make check

Copy link
Member

@ljflores ljflores left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Jayaprakash-ibm, it looks like there is an teuthology failure related to this change:

/a/yuriw-2025-01-13_19:12:38-rados-wip-yuri2-testing-2025-01-13-0620-distro-default-smithi/8074555

2025-01-13T23:03:29.787 INFO:teuthology.orchestra.run.smithi143.stdout:[ RUN      ] ObjectStore/StoreTest.mergeRegionTest/1
2025-01-13T23:03:32.632 INFO:teuthology.orchestra.run.smithi143.stderr:Creating object #-1:d3d08989:::Object:head#
2025-01-13T23:03:32.849 INFO:teuthology.orchestra.run.smithi143.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-6846-gacd9dd43/rpm/el9/BUILD/ceph-19.3.0-6846-gacd9dd43/src/include/interval_set.h: In function 'void interval_set<T, C>::erase(T, T, std::function<bool(T, T)>) [with T = long unsigned int; C = std::map]' thread 7f7793af8640 time 2025-01-13T23:03:32.848429+0000
2025-01-13T23:03:32.850 INFO:teuthology.orchestra.run.smithi143.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-6846-gacd9dd43/rpm/el9/BUILD/ceph-19.3.0-6846-gacd9dd43/src/include/interval_set.h: 567: FAILED ceph_assert(p != m.end())
2025-01-13T23:03:32.850 INFO:teuthology.orchestra.run.smithi143.stderr: ceph version 19.3.0-6846-gacd9dd43 (acd9dd43680a34295c804e0a31dea6a6f88a10ec) squid (dev)
2025-01-13T23:03:32.850 INFO:teuthology.orchestra.run.smithi143.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x11e) [0x7f779678c63c]
2025-01-13T23:03:32.850 INFO:teuthology.orchestra.run.smithi143.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x18c7fb) [0x7f779678c7fb]
2025-01-13T23:03:32.850 INFO:teuthology.orchestra.run.smithi143.stderr: 3: ceph_test_objectstore(+0x48832a) [0x5589a7d0132a]
2025-01-13T23:03:32.850 INFO:teuthology.orchestra.run.smithi143.stderr: 4: (KernelDevice::_aio_log_finish(IOContext*, unsigned long, unsigned long)+0xbc) [0x5589a7e7a4bc]
2025-01-13T23:03:32.850 INFO:teuthology.orchestra.run.smithi143.stderr: 5: (KernelDevice::_aio_thread()+0x127) [0x5589a7e7a7d7]
2025-01-13T23:03:32.850 INFO:teuthology.orchestra.run.smithi143.stderr: 6: ceph_test_objectstore(+0x5f3751) [0x5589a7e6c751]
2025-01-13T23:03:32.850 INFO:teuthology.orchestra.run.smithi143.stderr: 7: /lib64/libc.so.6(+0x8a062) [0x7f7795e8a062]
2025-01-13T23:03:32.851 INFO:teuthology.orchestra.run.smithi143.stderr: 8: /lib64/libc.so.6(+0x10f0e0) [0x7f7795f0f0e0]
2025-01-13T23:03:32.851 INFO:teuthology.orchestra.run.smithi143.stderr:*** Caught signal (Aborted) **
2025-01-13T23:03:32.851 INFO:teuthology.orchestra.run.smithi143.stderr: in thread 7f7793af8640 thread_name:bstore_aio
2025-01-13T23:03:32.851 INFO:teuthology.orchestra.run.smithi143.stderr: ceph version 19.3.0-6846-gacd9dd43 (acd9dd43680a34295c804e0a31dea6a6f88a10ec) squid (dev)
2025-01-13T23:03:32.851 INFO:teuthology.orchestra.run.smithi143.stderr: 1: /lib64/libc.so.6(+0x3e930) [0x7f7795e3e930]
2025-01-13T23:03:32.851 INFO:teuthology.orchestra.run.smithi143.stderr: 2: /lib64/libc.so.6(+0x8bdac) [0x7f7795e8bdac]
2025-01-13T23:03:32.851 INFO:teuthology.orchestra.run.smithi143.stderr: 3: raise()
2025-01-13T23:03:32.851 INFO:teuthology.orchestra.run.smithi143.stderr: 4: abort()
2025-01-13T23:03:32.851 INFO:teuthology.orchestra.run.smithi143.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x178) [0x7f779678c696]
2025-01-13T23:03:32.851 INFO:teuthology.orchestra.run.smithi143.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x18c7fb) [0x7f779678c7fb]
2025-01-13T23:03:32.852 INFO:teuthology.orchestra.run.smithi143.stderr: 7: ceph_test_objectstore(+0x48832a) [0x5589a7d0132a]
2025-01-13T23:03:32.852 INFO:teuthology.orchestra.run.smithi143.stderr: 8: (KernelDevice::_aio_log_finish(IOContext*, unsigned long, unsigned long)+0xbc) [0x5589a7e7a4bc]
2025-01-13T23:03:32.852 INFO:teuthology.orchestra.run.smithi143.stderr: 9: (KernelDevice::_aio_thread()+0x127) [0x5589a7e7a7d7]
2025-01-13T23:03:32.852 INFO:teuthology.orchestra.run.smithi143.stderr: 10: ceph_test_objectstore(+0x5f3751) [0x5589a7e6c751]
2025-01-13T23:03:32.852 INFO:teuthology.orchestra.run.smithi143.stderr: 11: /lib64/libc.so.6(+0x8a062) [0x7f7795e8a062]
2025-01-13T23:03:32.852 INFO:teuthology.orchestra.run.smithi143.stderr: 12: /lib64/libc.so.6(+0x10f0e0) [0x7f7795f0f0e0]

Testing ref: https://tracker.ceph.com/issues/69508

@Jayaprakash-ibm Jayaprakash-ibm force-pushed the jaya-branch branch 2 times, most recently from 8a8d5fc to 8d534be Compare February 10, 2025 11:10
@ceph ceph deleted a comment from hcbwiz Feb 10, 2025
@ceph ceph deleted a comment from hcbwiz Feb 10, 2025
@ceph ceph deleted a comment from hcbwiz Feb 10, 2025
@ceph ceph deleted a comment from hcbwiz Feb 10, 2025
@ceph ceph deleted a comment from hcbwiz Feb 10, 2025
@Jayaprakash-ibm
Copy link
Contributor Author

jenkins test make check

@aclamk aclamk self-requested a review February 11, 2025 10:06
@aclamk aclamk added aclamk-testing-phoebe bluestore testing and removed aclamk-testing-nauvoo bluestore testing labels Feb 11, 2025
@amathuria
Copy link
Contributor

@Jayaprakash-ibm Jayaprakash-ibm merged commit 3008ab4 into ceph:main Feb 13, 2025
12 checks passed
@Jayaprakash-ibm Jayaprakash-ibm deleted the jaya-branch branch February 13, 2025 16:24
@aclamk
Copy link
Contributor

aclamk commented Feb 24, 2025

Jayaprakash-ibm added a commit to Jayaprakash-ibm/ceph that referenced this pull request Feb 26, 2025
Sometimes Devices can be 0-sized, where I/O is impossible.

This change skips _aio_start() in open() if the device size is 0. This avoids unnecessary AIO setup.

This issue was introduced in PR : ceph#60791

Signed-off-by: Jaya Prakash <[email protected]>
Jayaprakash-ibm added a commit to Jayaprakash-ibm/ceph that referenced this pull request Feb 26, 2025
Sometimes devices can be 0-sized, where I/O is impossible.

This change skips _aio_start() in open() if the device size is 0. This avoids unnecessary AIO setup.

This issue was introduced in PR : ceph#60791

Signed-off-by: Jaya Prakash <[email protected]>
Jayaprakash-ibm added a commit to Jayaprakash-ibm/ceph that referenced this pull request Feb 27, 2025
Sometimes devices can be 0-sized, where I/O is impossible.

This change skips _aio_start() in open() if the device size is 0. This avoids unnecessary AIO setup.

This issue was introduced in PR : ceph#60791

Signed-off-by: Jaya Prakash <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants