build (new arch support) : Upgrade some PyPI dependency versions to enable better installation on riscv64#65142
build (new arch support) : Upgrade some PyPI dependency versions to enable better installation on riscv64#65142
Conversation
phlogistonjohn
left a comment
phlogistonjohn
left a comment
There was a problem hiding this comment.
Your Co-authored-by lines seem malformed. It should be adjacent to the Signed-off-by line and there should be a dash between authored and by. Also please note the patch submission guidelines and section about a proper signed off by line: https://github.com/ceph/ceph/blob/main/SubmittingPatches.rst
ffgan
force-pushed
the
feature/allow_riscv_install_deps
branch
from
dc1283e to
9eece0e
Compare
|
This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days. |
|
Can one of the admins verify this patch? |
…n riscv64 Co-authored by: [email protected]; Signed-off-by: ffgan <[email protected]>
ffgan
force-pushed
the
feature/allow_riscv_install_deps
branch
from
9eece0e to
4f5ebcf
Compare
ffgan
changed the title
./install-deps.sh to Execute on Ubuntu 22 + riscv64
|
Hi @phlogistonjohn . I've reorganized this PR, Welcome to review again. |
|
This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days. |
3 participants
Contribution Guidelines
To sign and title your commits, please refer to Submitting Patches to Ceph.
If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.
When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an
xbetween the brackets:[x]. Spaces and capitalization matter when checking off items this way.Checklist
Show available Jenkins commands
jenkins test classic perfJenkins Job | Jenkins Job Definitionjenkins test crimson perfJenkins Job | Jenkins Job Definitionjenkins test signedJenkins Job | Jenkins Job Definitionjenkins test make checkJenkins Job | Jenkins Job Definitionjenkins test make check arm64Jenkins Job | Jenkins Job Definitionjenkins test submodulesJenkins Job | Jenkins Job Definitionjenkins test dashboardJenkins Job | Jenkins Job Definitionjenkins test dashboard cephadmJenkins Job | Jenkins Job Definitionjenkins test apiJenkins Job | Jenkins Job Definitionjenkins test docsReadTheDocs | Github Workflow Definitionjenkins test ceph-volume allJenkins Jobs | Jenkins Jobs Definitionjenkins test windowsJenkins Job | Jenkins Job Definitionjenkins test rook e2eJenkins Job | Jenkins Job DefinitionThis PR has been reorganized and now simply upgrades the Python dependencies used by Ceph to make it easier to run
install-deps.shonFedora 41+riscv64.Since a certain number of projects still lack riscv64 support, if we want to add riscv64 support to Ceph, we need to do some additional work.
Let me re-introduce how to set up Ceph's dependencies in a
Fedora 41+riscv64environment. This mainly consists of two parts: RPM packages and PyPI packages.For RPM packages, thanks to Fedora's good riscv64 support, only two packages are missing:
golang-github-prometheusandlibpmem-devel(andlibpmemobj-devel,which together include thenvmlproject). I've completed the RPM packaging for these two packages in advance and stored the files here. If you're interested in the RPM build process, feel free to @ me - I'm happy to answer any questions. In the future, I'll try to contact the package maintainers to ask about adding riscv64 support for these two packages. But for now at least, we need to manually download and install the RPM packages. Since this operation is only temporary and will become unnecessary once official riscv64 support is obtained, I haven't included this operation in the current PR. This means that in the case ofFedora 41+riscv64, we need to manually resolve these two dependency issues before runninginstall-deps.sh.For PyPI packages, I'd like to divide this into two parts.
The first part consists of packages that already support riscv64. This part isn't reflected in this PR because when we use the latest versions of certain packages, these packages have already gained riscv64 support, so we don't need to worry about them. For example, packages like
rpds-pyandcoverage.The second part consists of packages that don't yet support riscv64, which is exactly what this PR introduces. These packages are supported by RISE's
wheel_builderproject - see details here. To use packages from wheel_builder, we need to add an additional PyPI source: https://gitlab.com/api/v4/projects/56254198/packages/pypi/simple. Similarly, since this operation is temporary and will no longer be needed as upstream fully supports riscv64 over time, I haven't included the source configuration operation in this PR either.After completing the setup for these two parts, we can easily build Ceph's dependency environment on
Fedora 41+riscv64. For specific setup steps, you can refer to my CI.So obviously, if upstream projects fully support riscv64 in the future, we'll need to open another PR similar to this one, which would also simply upgrade the dependency versions. Therefore, I believe the current PR should work well and won't introduce any temporary code.
I ran a simple test in my own fork, and the results show everything went smoothly. You can refer to this link.
(wheel_builder's support is still not quite complete, which causes some packages to need to be built from source, resulting in somewhat lengthy build times. I'm currently trying to resolve this issue. This issue won't affect the current PR.)
If you have any questions about the above content, feel free to @ me directly - I'm happy to answer any questions.
Other Info
Co-authored-by: Jincheng Ni [email protected]
Co-authored-by: kotvaer [email protected]