Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

clang-12 + Release mode causes zlib failures (apparent data corruption) #39313

Closed
djmarcin opened this issue Jul 8, 2021 · 11 comments
Closed

clang-12 + Release mode causes zlib failures (apparent data corruption) #39313

djmarcin opened this issue Jul 8, 2021 · 11 comments
Labels
build Issues and PRs related to build files or the CI. zlib Issues and PRs related to the zlib subsystem.

Comments

@djmarcin
Copy link

djmarcin commented Jul 8, 2021

Version

16.3.0, 16.4.0, 16.4.2

Platform

Linux 5.8.0-53-generic #60~20.04.1-Ubuntu SMP Thu May 6 09:52:46 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Subsystem

zlib

What steps will reproduce the bug?

Install clang 12.

$ sudo apt install clang-12
$ clang-12 --version
Ubuntu clang version 12.0.1-++20210605063018+0826268d59c6-1~exp1~20210605043737.99
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

Download and build Node with clang-12

$ curl -sL https://github.com/nodejs/node/archive/refs/tags/v16.4.2.tar.gz | tar xz
$ cd node-16.4.2 && export CC="clang-12" CXX="clang++-12" && ./configure && make -j$(nproc) BUILDTYPE=Release V=0 -C out

Run the following script with the produced binary:

$ curl -o /tmp/prettier.tgz https://registry.npmjs.org/prettier/-/prettier-2.2.1.tgz
$ cat - << EOF | out/Release/node
const fs = require('fs');
const { createGunzip } = require('zlib');
const { pipeline } = require('stream');
let inFile = fs.createReadStream('/tmp/prettier.tgz');
let outFile = fs.createWriteStream('/tmp/prettier.tar');
pipeline(inFile, createGunzip(), outFile, (err) => {
  if (err) {
    console.error('An error occurred:', err);
    process.exitCode = 1;
  }
});
EOF

Observe output:

An error occurred: Error: incorrect data check
    at Zlib.zlibOnError [as onerror] (node:zlib:191:17) {
  errno: -3,
  code: 'Z_DATA_ERROR'
}

How often does it reproduce? Is there a required condition?

Always when compilied in Release mode with clang-12

What is the expected behavior?

prettier.tar should be produced without errors

What do you see instead?

An error occurred: Error: incorrect data check
    at Zlib.zlibOnError [as onerror] (node:zlib:191:17) {
  errno: -3,
  code: 'Z_DATA_ERROR'
}

Additional information

This error does not appear when:

  • Compiling with BUILDTYPE=Debug and using out/Debug/node
  • Compiling with clang-11 (tested with Ubuntu clang version 11.0.0-2~ubuntu20.04.1)
  • Compiling with GCC (tested with gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0)

This error is also not present when compiling upstream zlib (https://zlib.net/) with clang-12.

This issue breaks npm install for prettier and several other packages as well because the files cannot be decompressed.

Built on an AMD Threadripper CPU.
@himself65 himself65 added build Issues and PRs related to build files or the CI. macos Issues and PRs related to the macOS platform / OSX. and removed macos Issues and PRs related to the macOS platform / OSX. labels Jul 9, 2021
@jesec
Copy link
Member

jesec commented Aug 20, 2021

Refs: vercel/pkg-fetch#208

I can reproduce this issue with Apple Clang compiled macos-x64 executable.

clang -v
Apple clang version 13.0.0 (clang-1300.0.29.3)
Target: arm64-apple-darwin21.0.0
Thread model: posix
InstalledDir: /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

./configure && make

At least Node 14 and 16 release lines are impacted.

Interestingly, the issue does not appear with arm64 executable compiled with the same toolchain.

This could become a release-critical bug, once the new Clang/Xcode versions roll in. I suggest that we re-triage this issue as zlib, confirmed-bug.

I am looking at the issue right now, and to raise attention here are some pings to recent committers to the zlib parts: @RaisinTen, @addaleax, @mscdex, @aduh95, @ronag .

@RaisinTen
Copy link
Contributor

Perhaps we need to apply the fix referred to in #31800 (comment) to solve this issue. But we need #33044 to land first to be able to update our copy of zlib.

@RaisinTen RaisinTen added the zlib Issues and PRs related to the zlib subsystem. label Aug 22, 2021
jesec added a commit to vercel/pkg-fetch that referenced this issue Aug 24, 2021
@jesec
build: workaround nodejs/node#39313 for macOS
925111b 
All supported macOS versions have zlib as a system library.
@jesec jesec mentioned this issue Sep 6, 2021
Merged
igrayson added a commit to igrayson/nvm that referenced this issue Nov 15, 2021
@igrayson
Update README.md
1eb4d8b 
Describe another failure case for installing node versions from source on mac arm64 machines.

It seems to be related to nodejs/node#39313 (which describes a similar issue).
@igrayson igrayson mentioned this issue Nov 15, 2021
Merged
ljharb pushed a commit to igrayson/nvm that referenced this issue Nov 19, 2021
@igrayson @ljharb
[readme] Describe another failure case for installing node versions f…
e9b2693 
…rom source on mac arm64 machines.

It seems to be related to nodejs/node#39313 (which describes a similar issue).
@convenient convenient mentioned this issue Dec 1, 2021
Closed
@meta-github
Copy link

meta-github commented Jan 16, 2022 •
edited
Loading

Newer version of clang (tested on macOS with Xcode 13.2.1) seem to fail to generate correct code when compiling zlib/contrib/optimizations/inffast_chunk.c with -O2 or higher optimization. -O1 seems to work fine.

@RaisinTen RaisinTen mentioned this issue Jan 30, 2022
Closed
@lpinca
Copy link
Member

lpinca commented Nov 17, 2022

#45387 landed. I cannot reproduce on macOS with clang 14. Can this be closed?

@meta-github
Copy link

Did the fix find its way into all currently supported versions of node.js?

@lpinca
Copy link
Member

lpinca commented Nov 17, 2022

Did the fix find its way into all currently supported versions of node.js?

#45387 is not yet included in any version.

@meta-github
Copy link

I suggest to close the bug after the upstream fix has been merged into all currently supported versions of node.js.

@lpinca
Copy link
Member

lpinca commented Nov 17, 2022

Why? we usually close the issue when the fix lands on main. If the issue is fixed on main there is no reason to keep it open.

@meta-github
Copy link

I can confirm that the bug no longer exists on main. So, you might close this if you want. I still need to keep my workaround in my build script for releases until all releases are fixed.

@lpinca
Copy link
Member

lpinca commented Nov 18, 2022

Thank you for the confirmation.

@lpinca lpinca closed this as completed Nov 18, 2022
@fabOnReact
Copy link

fabOnReact commented Apr 30, 2023 •
edited
Loading

I experience this issue with clang-14, nvm, node 16.15.1, npm 8.11.0, macOS 13.3.1 (22E261). Using nvm install v16.15.1 --shared-zlib fixed the issue. npm install would fail with Error code Z_DATA_ERROR Zlib: incorrect data check -3. I followed this instructions to fix it

@danopia danopia mentioned this issue Aug 23, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
build Issues and PRs related to build files or the CI. zlib Issues and PRs related to the zlib subsystem.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@djmarcin @lpinca @meta-github @jesec @himself65 @fabOnReact @RaisinTen