Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alpine 3.19 + Java 8: Symbol not found: __xstat64 #7455

Closed
bratkartoffel opened this issue Dec 12, 2023 · 8 comments · Fixed by sbt/io#362
Closed

Alpine 3.19 + Java 8: Symbol not found: __xstat64 #7455

bratkartoffel opened this issue Dec 12, 2023 · 8 comments · Fixed by sbt/io#362
Labels

Comments

@bratkartoffel
Copy link

bratkartoffel commented Dec 12, 2023

steps

Running sbt with java 8 on alpine 3.19 fails due to java.lang.UnsatisfiedLinkError.
Try to invoke sbt when running on alpine 3.19, e.g. using docker:

docker run --rm -it alpine:3.19
# inside the container:
apk add openjdk8 bash
cd $(mktemp -d)
# download sbt
wget https://github.com/sbt/sbt/releases/download/v1.9.7/sbt-1.9.7.tgz
tar -xzf sbt-1.9.7.tgz
# create example project (crash)
yes "" | sbt/bin/sbt new scala/hello-world.g8

problem

sbt can't be run when using java 8 and latest musl libc. All invocations fail with the following stacktrace:

/tmp/tmp.aJhdoP # yes "" | sbt/bin/sbt new scala/hello-world.g8
[info] [launcher] getting org.scala-sbt sbt 1.9.7  (this may take some time)...
[info] [launcher] getting Scala 2.12.18 (for sbt)...
java.lang.UnsatisfiedLinkError: Error looking up function '__xstat64': Symbol not found: __xstat64
        at com.sun.jna.Function.<init>(Function.java:252)
        at com.sun.jna.NativeLibrary.getFunction(NativeLibrary.java:620)
        at com.sun.jna.NativeLibrary.getFunction(NativeLibrary.java:596)
        at com.sun.jna.NativeLibrary.getFunction(NativeLibrary.java:582)
        at com.sun.jna.Library$Handler.invoke(Library.java:248)
        at com.sun.proxy.$Proxy0.__xstat64(Unknown Source)
        at sbt.internal.io.Linux64Milli$.$anonfun$getModifiedTimeNative$1(Milli.scala:215)
        at sbt.internal.io.MilliPosixBase.checkedIO(Milli.scala:100)
        at sbt.internal.io.Linux64Milli$.getModifiedTimeNative(Milli.scala:215)
        at sbt.internal.io.Linux64Milli$.getModifiedTimeNative(Milli.scala:210)
        at sbt.internal.io.MilliNative.getModifiedTime(Milli.scala:65)
        at sbt.internal.io.Milli$.getModifiedTime(Milli.scala:387)
        at sbt.io.IO$.$anonfun$getModifiedTimeOrZero$1(IO.scala:1436)
        at sbt.internal.io.Retry$.apply$mJc$sp(Retry.scala:47)
        at sbt.internal.io.Retry$.apply$mJc$sp(Retry.scala:29)
        at sbt.internal.io.Retry$.apply$mJc$sp(Retry.scala:24)
        at sbt.io.IO$.getModifiedTimeOrZero(IO.scala:1436)
        at sbt.internal.classpath.ClassLoaderCache$Key$$anonfun$$lessinit$greater$3.apply(ClassLoaderCache.scala:59)
        at sbt.internal.classpath.ClassLoaderCache$Key$$anonfun$$lessinit$greater$3.apply(ClassLoaderCache.scala:59)
        at scala.collection.immutable.List.map(List.scala:293)
        at sbt.internal.classpath.ClassLoaderCache$Key.<init>(ClassLoaderCache.scala:59)
        at sbt.internal.classpath.ClassLoaderCache$Key.<init>(ClassLoaderCache.scala:60)
        at sbt.internal.classpath.ClassLoaderCache.cachedCustomClassloader(ClassLoaderCache.scala:195)
        at sbt.State$StateOpsImpl$.initializeClassLoaderCache$extension(State.scala:417)
        at sbt.StandardMain$.initialState(Main.scala:286)
        at sbt.xMain$.$anonfun$run$9(Main.scala:112)
        at sbt.io.IO$.withTemporaryDirectory(IO.scala:496)
        at sbt.io.IO$.withTemporaryDirectory(IO.scala:506)
        at sbt.xMain$.run(Main.scala:102)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at sbt.internal.XMainConfiguration.run(XMainConfiguration.java:59)
        at sbt.xMain.run(Main.scala:47)
        at xsbt.boot.Launch$.$anonfun$run$1(Launch.scala:149)
        at xsbt.boot.Launch$.withContextLoader(Launch.scala:176)
        at xsbt.boot.Launch$.run(Launch.scala:149)
        at xsbt.boot.Launch$.$anonfun$apply$1(Launch.scala:44)
        at xsbt.boot.Launch$.launch(Launch.scala:159)
        at xsbt.boot.Launch$.apply(Launch.scala:44)
        at xsbt.boot.Launch$.apply(Launch.scala:21)
        at xsbt.boot.Boot$.runImpl(Boot.scala:78)
        at xsbt.boot.Boot$.run(Boot.scala:73)
        at xsbt.boot.Boot$.main(Boot.scala:21)
        at xsbt.boot.Boot.main(Boot.scala)
[error] [launcher] error during sbt launcher: java.lang.UnsatisfiedLinkError: Error looking up function '__xstat64': Symbol not found: __xstat64

expectation

Builds using java 8 on latest alpine release should be possible.

notes

When using Java 11 (openjdk11-jdk) / Java 17 (openjdk17-jdk) / Java 21 (openjdk21-jdk), everything is fine.
When using alpine:3.18, everything is fine too.

@eed3si9n
Copy link
Member

Thanks for the report. I think a workaround would be setting -Dsbt.io.jdktimestamps=true - https://github.com/sbt/io/blob/2cc96a8295be4ae2086d34ed89c735adf6b27dbb/io/src/main/scala/sbt/internal/io/Milli.scala#L356-L360, but doing so would likely break incremental compilation because part of the code uses time as invalidation, and time increment isn't granular enough for incremental compilation.

@bratkartoffel
Copy link
Author

In my case this is workaround is fine as the build is running on a cicd server, so incremental compilation is no issue there. My local environment is already running on java 21, so this is also no problem.

But, as far as I can see from the source, the usage of _xstat64() is not correct?
LSB 5.0 (https://refspecs.linuxbase.org/LSB_5.0.0/LSB-Core-generic/LSB-Core-generic/baselib---xstat64.html) states that "The behavior of these functions for values of ver other than _STAT_VER is undefined.", but you call it with a "1" for Linux32 (https://github.com/sbt/io/blob/2cc96a8295be4ae2086d34ed89c735adf6b27dbb/io/src/main/scala/sbt/internal/io/Milli.scala#L215) or "3" for Linux64 (https://github.com/sbt/io/blob/2cc96a8295be4ae2086d34ed89c735adf6b27dbb/io/src/main/scala/sbt/internal/io/Milli.scala#L229)
So depending on the system, the value of _STAT_VER might be different, leading to undefined behaviour.

Why don't you use stat64() instead?

@eed3si9n
Copy link
Member

Why don't you use stat64() instead?

Honestly I'm not sure since I did review the change at the time, but did not catch the nuance of the native call that was chosen. If I were to venture a guess this was an attempt to have a stable runtime behavior on Ubuntu by hardcoding to a specific version? I'm also a bit confused since as far as I know sbt has been used inside Alpine before.

@bratkartoffel
Copy link
Author

bratkartoffel commented Dec 13, 2023

The difference is in the different musl libc.
Before alpine:3.19:

/ # apk add binutils
/ # readelf -s /lib/ld-musl-x86_64.so.1 | grep xstat
   241: 00000000000495e3    16 FUNC    WEAK   DEFAULT    9 __fxstatat64
   331: 00000000000495f3    11 FUNC    GLOBAL DEFAULT    9 __lxstat
   735: 00000000000495d9    10 FUNC    WEAK   DEFAULT    9 __fxstat64
   933: 00000000000495f3    11 FUNC    WEAK   DEFAULT    9 __lxstat64
   969: 00000000000495fe    11 FUNC    WEAK   DEFAULT    9 __xstat64
  1137: 00000000000495d9    10 FUNC    GLOBAL DEFAULT    9 __fxstat
  1496: 00000000000495fe    11 FUNC    GLOBAL DEFAULT    9 __xstat
  1564: 00000000000495e3    16 FUNC    GLOBAL DEFAULT    9 __fxstatat

Starting with alpine:3.19:

/ # apk add binutils
/ # readelf -s /lib/ld-musl-x86_64.so.1 | grep xstat
   321: 000000000004def4    11 FUNC    GLOBAL DEFAULT   11 __lxstat
  1099: 000000000004deda    10 FUNC    GLOBAL DEFAULT   11 __fxstat
  1442: 000000000004deff    11 FUNC    GLOBAL DEFAULT   11 __xstat
  1508: 000000000004dee4    16 FUNC    GLOBAL DEFAULT   11 __fxstatat

I've found some commits mentioning LFS64 specific changes in the musl-libc at alpine, but I don't really understand why it was removed. I'll ask at the irc and post the response here.

// Edit: LFS64 support was removed from musl, but alpine decided to add a workaround in older releases. This workaround was removed with alpine:3.19, so the issue arises now. Details why it was removed and why it shouldn't be used in first place can be found here: https://www.openwall.com/lists/musl/2022/09/26/1

@bratkartoffel
Copy link
Author

bratkartoffel commented Dec 13, 2023

After reading the thread at the musl mailinglist I come to the conclusion that using a non standard-abi like the lfs64 stuff is the root cause of the problem here. So stat64() wouldn't be a solution as it's also missing in newer alpine musl libc. Can't you just use stat() instead? The stat struct defines the st_ctime (which is the relevant part here) as a struct timespec, supporting nanoseconds since linux kernel 2.5.48, dating back to 2002.

Test application:

#include <stdio.h>
#include <sys/stat.h>

int main(void) {
        struct stat statbuf;
        printf("stat() = %i\n", stat("/etc/os-release", &statbuf));
        printf("sizeof(st_ctim) = %zu\n", sizeof(statbuf.st_ctim));
        printf("st_ctim.tv_nsec = %li\n", statbuf.st_ctim.tv_nsec);
        printf("st_ctim.tv_sec = %li\n", statbuf.st_ctim.tv_sec);
}

On alpine x86_64:

stat() = 0
sizeof(st_ctim) = 16
st_ctim.tv_nsec = 387540266
st_ctim.tv_sec = 1702358349

On alpine x86:

stat() = 0
sizeof(st_ctim) = 16
st_ctim.tv_nsec = 249564841
st_ctim.tv_sec = 1702475428

On alpine armhf:

stat() = 0
sizeof(st_ctim) = 16
st_ctim.tv_nsec = 831276549
st_ctim.tv_sec = 1073743964

On Ubuntu 22.04 armv7:

stat() = 0
sizeof(st_ctim) = 8
st_ctim.tv_nsec = 747936871
st_ctim.tv_sec = 1702475752

On Ubuntu 18.04 x86:

stat() = 0
sizeof(st_ctim) = 8
st_ctim.tv_nsec = 36894514
st_ctim.tv_sec = 1702475910

So using just stat() should be fine as it provides timestamps with nanoseconds precision.

Doc:

@eed3si9n
Copy link
Member

Sounds good to me. Would you like to send a pull request for this?

@bratkartoffel
Copy link
Author

PR is open. If you could build me a version of sbt including this fix, I can test it on the various architectures / systems I've mentioned in #7455 (comment)

@eed3si9n
Copy link
Member

t83714 added a commit to magda-io/magda that referenced this issue Dec 28, 2023
eed3si9n added a commit to eed3si9n/io that referenced this issue Feb 23, 2024
**Problem**
Ref sbt/sbt#7463
Ref sbt/sbt#7455

On some Linux sbt 1.9.8 fail with "java.lang.UnsatisfiedLinkError: Error looking up function 'stat'".
This is due to our workaround another stat related-issue.
The general issue is that we currently call native call to retrieve
the last modified time because JDK 8 use to have a bug JDK-8177809
that failed to get milliseconds.
The bug was fixed in 2021 openjdk8u302.

**Solution**
We can drop the whole native code mechanism.
eed3si9n added a commit to eed3si9n/io that referenced this issue Feb 23, 2024
**Problem**
Ref sbt/sbt#7463
Ref sbt/sbt#7455

On some Linux sbt 1.9.8 fail with "java.lang.UnsatisfiedLinkError: Error looking up function 'stat'".
This is due to our workaround another stat related-issue.
The general issue is that we currently call native call to retrieve
the last modified time because JDK 8 use to have a bug JDK-8177809
that failed to get milliseconds.
The bug was fixed in 2021 openjdk8u302.

**Solution**
We can drop the whole native code mechanism.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants