[dylink] Fix rpath calculation in nested dependencies by ryanking13 · Pull Request #24234 · emscripten-core/emscripten

ryanking13 · 2025-05-01T10:52:47Z

This fixes a bug in rpath calculation in nested dependencies.

Basically, it fixes a case when a relative path is passed to the loadDynamicLibrary.

Think of the following scenario:

libhello.wasm depends on libhello_dep.so depends on libhello_nested_dep.so
[main.c] calls dlopen('/usr/lib/libhello.wasm')
[dynlink.c] calls loadDynamicLibrary('/usr/lib/libhello.wasm') (_dlopen_js ==> dlopenInternal ==> loadDynamicLibrary)
[libdylink.js] inside loadDynamicLibrary('/usr/lib/libhello.wasm'), it loads the dependency libhello_dep.so. When locating libhello_dep.so, it uses parentLibPath: /usr/lib/libhello.wasm it locates the library correctly from /usr/lib/libhello_dep.so.
[libdylink.js] Now we call loadDynamicLibrary('libhello_dep.so'), and it loads its dependency 'libhello_nested_dep.so' first. In this case it uses parentLibPath: libhello_dep.so so it fails to locate the library.

This is an automatic change generated by tools/maint/rebaseline_tests.py. The following (2) test expectation files were updated by running the tests with `--rebaseline`: ``` other/codesize/test_codesize_hello_dylink.gzsize: 11735 => 11758 [+23 bytes / +0.20%] other/codesize/test_codesize_hello_dylink.jssize: 27774 => 27799 [+25 bytes / +0.09%] Average change: +0.14% (+0.09% - +0.20%) ```

sbc100 · 2025-05-01T14:28:35Z

src/lib/libdylink.js

  $newDSO: (name, handle, syms) => {
    var dso = {
      refcount: Infinity,
      name,


So does that mean that name is always just the basename / so_name? Is is name sometimes absolute too here?

No, only the dependencies of the libraries will have only the so_name in name. The parent library called from dlopen would have absolute path in the name.

No, only the dependencies of the libraries will have only the so_name in name.

So, I think the alternative approach would be ensuring loadDynamicLibrary always take absolute path. Let me try that approach instead.

test/test_other.py

src/lib/libdylink.js

sbc100 · 2025-05-01T14:36:49Z

src/lib/libdylink.js

-    flags = {...flags, rpath: { parentLibPath: libName, paths: metadata.runtimePaths }}
+    var dso = LDSO.loadedLibsByName[libName];
+    var libPath = dso?.path ?? libName;
+    flags = {...flags, rpath: { parentLibPath: libPath, paths: metadata.runtimePaths }}


Should parentLibPath be just the dirname part of the parent? i.e. is there any point in passing a value if its just a basename?

I think it is possible to calculate the dirname here.

For now, the dirname is calculated inside replaceORIGIN function, and the reason I did that was to isolate the code that depends on FS or PATH as much as possible.

If we calculate the dirname here, I think it can be something like:

var dso = LDSO.loadedLibsByName[libName]; #if FILESYSTEM var libPath = PATH.dirname(dso?.path ?? libName); #endif flags = {...flags, rpath: { parentLibPath: libPath, paths: metadata.runtimePaths }}

This is an automatic change generated by tools/maint/rebaseline_tests.py. The following (2) test expectation files were updated by running the tests with `--rebaseline`: ``` other/codesize/test_codesize_hello_dylink.gzsize: 11758 => 11757 [-1 bytes / -0.01%] other/codesize/test_codesize_hello_dylink.jssize: 27799 => 27823 [+24 bytes / +0.09%] Average change: +0.04% (-0.01% - +0.09%) ```

This reverts commit af36541.

ryanking13 · 2025-06-02T15:25:24Z

@sbc100, May I ask for your review again when you have time?

sbc100 · 2025-11-06T20:46:57Z

@ryanking13 I'm curious about the different between this patch and #25694. Do they both solve the same issue? Do you have a preference for one over the other?

sbc100 · 2025-11-06T20:47:17Z

Also, could you rebase (or merge) this ?

ryanking13 · 2025-11-08T07:06:38Z

@ryanking13 I'm curious about the different between this patch and #25694. Do they both solve the same issue? Do you have a preference for one over the other?

@sbc100 Thanks for the review!

I think both approaches somehow solve this nested dependency issue, but I prefer mine because I think this is more straightforward. #25694 modifies the libName variable in multiple places which increases the cognitive load. This PR always resolves the libname before running loadDynamicLibrary soi I think it is better.

This is an automatic change generated by tools/maint/rebaseline_tests.py. The following (2) test expectation files were updated by running the tests with `--rebaseline`: ``` codesize/test_codesize_hello_dylink.json: 44349 => 44349 [+0 bytes / +0.00%] codesize/test_codesize_hello_dylink_all.json: 819566 => 819566 [+0 bytes / +0.00%] Average change: +0.00% (+0.00% - +0.00%) ```

sbc100

Would this also fix #25814?

ryanking13 · 2025-11-20T14:32:46Z

@sbc100

Would this also fix #25814?

Unfortunately no, I tested the worker case but I get fiile not found error. Does --embed-file parameter work differently in worker?

This is an automatic change generated by tools/maint/rebaseline_tests.py. The following (30) test expectation files were updated by running the tests with `--rebaseline`: ``` codesize/test_codesize_cxx_ctors1.json: 149124 => 149173 [+49 bytes / +0.03%] codesize/test_codesize_cxx_ctors2.json: 148528 => 148577 [+49 bytes / +0.03%] codesize/test_codesize_cxx_except.json: 194587 => 194636 [+49 bytes / +0.03%] codesize/test_codesize_cxx_except_wasm.json: 164104 => 164153 [+49 bytes / +0.03%] codesize/test_codesize_cxx_except_wasm_legacy.json: 161762 => 161811 [+49 bytes / +0.03%] codesize/test_codesize_cxx_lto.json: 125465 => 125448 [-17 bytes / -0.01%] codesize/test_codesize_cxx_mangle.json: 258658 => 258718 [+60 bytes / +0.02%] codesize/test_codesize_cxx_noexcept.json: 151541 => 151590 [+49 bytes / +0.03%] codesize/test_codesize_cxx_wasmfs.json: 176718 => 176912 [+194 bytes / +0.11%] codesize/test_codesize_files_wasmfs.json: 55679 => 55818 [+139 bytes / +0.25%] codesize/test_codesize_hello_O0.json: 39326 => 39362 [+36 bytes / +0.09%] codesize/test_codesize_hello_dylink.json: 44349 => 44469 [+120 bytes / +0.27%] codesize/test_codesize_hello_dylink_all.json: 819566 => 820006 [+440 bytes / +0.05%] codesize/test_codesize_mem_O3.json: 9612 => 9610 [-2 bytes / -0.02%] codesize/test_codesize_mem_O3_grow.json: 9898 => 9896 [-2 bytes / -0.02%] codesize/test_codesize_mem_O3_grow_standalone.json: 9651 => 9649 [-2 bytes / -0.02%] codesize/test_codesize_mem_O3_standalone.json: 9509 => 9507 [-2 bytes / -0.02%] codesize/test_codesize_mem_O3_standalone_lib.json: 8804 => 8802 [-2 bytes / -0.02%] codesize/test_codesize_mem_O3_standalone_narg.json: 8837 => 8835 [-2 bytes / -0.02%] codesize/test_codesize_mem_O3_standalone_narg_flto.json: 7649 => 7647 [-2 bytes / -0.03%] codesize/test_codesize_minimal_pthreads.json: 27226 => 27220 [-6 bytes / -0.02%] codesize/test_codesize_minimal_pthreads_memgrowth.json: 27654 => 27648 [-6 bytes / -0.02%] codesize/test_minimal_runtime_code_size_hello_webgl2_wasm.json: 13201 => 13204 [+3 bytes / +0.02%] codesize/test_minimal_runtime_code_size_hello_webgl2_wasm2js.json: 18539 => 18554 [+15 bytes / +0.08%] codesize/test_minimal_runtime_code_size_hello_webgl2_wasm_singlefile.json: 15136 => 15147 [+11 bytes / +0.07%] codesize/test_minimal_runtime_code_size_hello_webgl_wasm.json: 12739 => 12742 [+3 bytes / +0.02%] codesize/test_minimal_runtime_code_size_hello_webgl_wasm2js.json: 18066 => 18081 [+15 bytes / +0.08%] codesize/test_minimal_runtime_code_size_random_printf_wasm.json: 10980 => 10993 [+13 bytes / +0.12%] codesize/test_minimal_runtime_code_size_random_printf_wasm2js.json: 17169 => 17229 [+60 bytes / +0.35%] codesize/test_unoptimized_code_size.json: 180948 => 181056 [+108 bytes / +0.06%] Average change: +0.05% (-0.03% - +0.35%) ```

sbc100 · 2025-11-20T17:46:20Z

To get the codesize expectations to match you need to run toosl/main/rebaseline_test.py after first doing emsdk install tot. If you have having trouble doing that let me know and I can push the expected results.

This is an automatic change generated by tools/maint/rebaseline_tests.py. The following (2) test expectation files were updated by running the tests with `--rebaseline`: ``` codesize/test_codesize_cxx_lto.json: 125448 => 125450 [+2 bytes / +0.00%] codesize/test_unoptimized_code_size.json: 180908 => 181016 [+108 bytes / +0.06%] Average change: +0.03% (+0.00% - +0.06%) ```

This is an automatic change generated by tools/maint/rebaseline_tests.py. The following (26) test expectation files were updated by running the tests with `--rebaseline`: ``` codesize/test_codesize_cxx_ctors1.json: 149173 => 149124 [-49 bytes / -0.03%] codesize/test_codesize_cxx_ctors2.json: 148577 => 148528 [-49 bytes / -0.03%] codesize/test_codesize_cxx_except.json: 194636 => 194587 [-49 bytes / -0.03%] codesize/test_codesize_cxx_except_wasm.json: 164153 => 164104 [-49 bytes / -0.03%] codesize/test_codesize_cxx_except_wasm_legacy.json: 161811 => 161762 [-49 bytes / -0.03%] codesize/test_codesize_cxx_mangle.json: 258718 => 258658 [-60 bytes / -0.02%] codesize/test_codesize_cxx_noexcept.json: 151590 => 151541 [-49 bytes / -0.03%] codesize/test_codesize_cxx_wasmfs.json: 176912 => 176718 [-194 bytes / -0.11%] codesize/test_codesize_files_wasmfs.json: 55818 => 55679 [-139 bytes / -0.25%] codesize/test_codesize_hello_O0.json: 39362 => 39326 [-36 bytes / -0.09%] codesize/test_codesize_mem_O3.json: 9610 => 9612 [+2 bytes / +0.02%] codesize/test_codesize_mem_O3_grow.json: 9896 => 9898 [+2 bytes / +0.02%] codesize/test_codesize_mem_O3_grow_standalone.json: 9649 => 9651 [+2 bytes / +0.02%] codesize/test_codesize_mem_O3_standalone.json: 9507 => 9509 [+2 bytes / +0.02%] codesize/test_codesize_mem_O3_standalone_lib.json: 8802 => 8804 [+2 bytes / +0.02%] codesize/test_codesize_mem_O3_standalone_narg.json: 8835 => 8837 [+2 bytes / +0.02%] codesize/test_codesize_minimal_pthreads.json: 27220 => 27226 [+6 bytes / +0.02%] codesize/test_codesize_minimal_pthreads_memgrowth.json: 27648 => 27654 [+6 bytes / +0.02%] codesize/test_minimal_runtime_code_size_hello_webgl2_wasm.json: 13204 => 13201 [-3 bytes / -0.02%] codesize/test_minimal_runtime_code_size_hello_webgl2_wasm2js.json: 18554 => 18539 [-15 bytes / -0.08%] codesize/test_minimal_runtime_code_size_hello_webgl2_wasm_singlefile.json: 15147 => 15136 [-11 bytes / -0.07%] codesize/test_minimal_runtime_code_size_hello_webgl_wasm.json: 12742 => 12739 [-3 bytes / -0.02%] codesize/test_minimal_runtime_code_size_hello_webgl_wasm2js.json: 18081 => 18066 [-15 bytes / -0.08%] codesize/test_minimal_runtime_code_size_random_printf_wasm.json: 10993 => 10980 [-13 bytes / -0.12%] codesize/test_minimal_runtime_code_size_random_printf_wasm2js.json: 17229 => 17169 [-60 bytes / -0.35%] codesize/test_unoptimized_code_size.json: 181016 => 180908 [-108 bytes / -0.06%] Average change: -0.05% (-0.35% - +0.02%) ```

This is an automatic change generated by tools/maint/rebaseline_tests.py. The following (4) test expectation files were updated by running the tests with `--rebaseline`: ``` codesize/test_codesize_cxx_lto.json: 125450 => 125465 [+15 bytes / +0.01%] codesize/test_codesize_hello_dylink.json: 44469 => 44463 [-6 bytes / -0.01%] codesize/test_codesize_hello_dylink_all.json: 820006 => 819727 [-279 bytes / -0.03%] codesize/test_codesize_mem_O3_standalone_narg_flto.json: 7647 => 7649 [+2 bytes / +0.03%] Average change: -0.00% (-0.03% - +0.03%) ```

ryanking13 · 2025-11-21T12:02:47Z

@sbc100

To get the codesize expectations to match you need to run toosl/main/rebaseline_test.py after first doing emsdk install tot. If you have having trouble doing that let me know and I can push the expected results.

Thanks, updated the codesize. I think where was some code courrupted in my cache.

…e#24234) This fixes a bug in rpath calculation in nested dependencies. Basically, it fixes a case when a relative path is passed to the `loadDynamicLibrary`. Think of the following scenario: 0. libhello.wasm depends on libhello_dep.so depends on libhello_nested_dep.so 1. [main.c] calls dlopen('/usr/lib/libhello.wasm') 2. [dynlink.c] calls `loadDynamicLibrary('/usr/lib/libhello.wasm')` (_dlopen_js ==> dlopenInternal ==> loadDynamicLibrary) 3. [libdylink.js] inside `loadDynamicLibrary('/usr/lib/libhello.wasm')`, it loads the dependency `libhello_dep.so`. When locating `libhello_dep.so`, it uses `parentLibPath: /usr/lib/libhello.wasm` it locates the library correctly from `/usr/lib/libhello_dep.so`. 5. [libdylink.js] Now we call `loadDynamicLibrary('libhello_dep.so')`, and it loads its dependency 'libhello_nested_dep.so' first. In this case it uses `parentLibPath: libhello_dep.so` so it fails to locate the library.

ryanking13 added 4 commits May 1, 2025 10:37

Fix nested dependencies handling in rpath

742a3e6

fallback

f1f85ff

syntax

65f39d4

sbc100 changed the title ~~Fix rpath calculation in nested dependencies~~ [dylink] Fix rpath calculation in nested dependencies May 1, 2025

sbc100 added the dynamic_linking label May 1, 2025

sbc100 reviewed May 1, 2025

View reviewed changes

ryanking13 added 6 commits May 2, 2025 07:28

Merge remote-tracking branch 'upstream/main' into rpath-nested-deps

3fb1ddd

wasm ==> so

62dc4a4

Always pass absolute path in loadDynamicLibrary

81b1dbd

Remove unnecessary changes

f4e3b25

Merge branch 'main' into rpath-nested-deps

d9289c2

ryanking13 mentioned this pull request May 19, 2025

0.28 release pyodide/pyodide#5650

Closed

hoodmane requested review from sbc100 and removed request for sbc100 May 31, 2025 00:10

ryanking13 added 4 commits May 31, 2025 13:23

Merge remote-tracking branch 'upstream/main' into rpath-nested-deps

3396b43

Store filename only

af36541

Revert "Store filename only"

f4fecbb

This reverts commit af36541.

Fix codesize

b86e1f0

ryanking13 requested a review from sbc100 June 2, 2025 15:24

ryanking13 mentioned this pull request Nov 1, 2025

Fix nested dynamic library loading via RPATH #25694

Closed

kleisauke mentioned this pull request Nov 5, 2025

Unable to load nested dynamic library when loading side module #25722

Open

Merge remote-tracking branch 'upstream/main' into rpath-nested-deps

805749f

ryanking13 added 2 commits November 8, 2025 18:35

rebaseline

12f094a

sbc100 reviewed Nov 17, 2025

View reviewed changes

Merge remote-tracking branch 'upstream/main' into rpath-nested-deps

2813537

sbc100 approved these changes Nov 20, 2025

View reviewed changes

ryanking13 added 5 commits November 21, 2025 14:00

Merge remote-tracking branch 'upstream/main' into rpath-nested-deps

bd87116

Remove codesize files that are not used anymore

7fc0ca7

sbc100 approved these changes Nov 21, 2025

View reviewed changes

sbc100 merged commit fb4a23b into emscripten-core:main Nov 21, 2025
34 checks passed

ianthomas23 mentioned this pull request Nov 25, 2025

Experiment building scipy using openblas-flang emscripten-forge/recipes#3905

Closed

ryanking13 mentioned this pull request Nov 28, 2025

Fix nested dynamic library dependencies + RPATH pyodide/pyodide#5981

Closed

2 tasks

pepijndevos mentioned this pull request Mar 6, 2026

[dylink] Normalize library paths to prevent duplicate loading #26399

Open

Conversation

ryanking13 commented May 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sbc100 May 1, 2025

Choose a reason for hiding this comment

Uh oh!

ryanking13 May 2, 2025

Choose a reason for hiding this comment

Uh oh!

ryanking13 May 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

sbc100 May 1, 2025

Choose a reason for hiding this comment

Uh oh!

ryanking13 May 2, 2025

Choose a reason for hiding this comment

Uh oh!

ryanking13 commented Jun 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sbc100 commented Nov 6, 2025

Uh oh!

sbc100 commented Nov 6, 2025

Uh oh!

ryanking13 commented Nov 8, 2025

Uh oh!

sbc100 left a comment

Choose a reason for hiding this comment

Uh oh!

ryanking13 commented Nov 20, 2025

Uh oh!

sbc100 commented Nov 20, 2025

Uh oh!

ryanking13 commented Nov 21, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ryanking13 commented May 1, 2025 •

edited

Loading

ryanking13 May 2, 2025 •

edited

Loading

ryanking13 commented Jun 2, 2025 •

edited

Loading