Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(terser): always make at least 1 worker thread #1604

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

easrng
Copy link

@easrng easrng commented Oct 8, 2023

Rollup Plugin Name: terser

This PR contains:

  • bugfix
  • feature
  • refactor
  • documentation
  • other

Are tests included?

  • yes (bugfixes and features will not be merged without tests)
  • no

Breaking Changes?

  • yes (breaking changes will not be merged unless absolutely necessary)
  • no

If yes, then include "BREAKING CHANGES:" in the first commit message body, followed by a description of what is breaking.

List any relevant issue numbers:
Resolves #1594

Description

On some platforms (I noticed it on Android), os.cpus() can return [], resulting in no workers being spawned, so the jobs never run. This patch makes sure the maximum number of workers is never 0.

@easrng easrng requested a review from tada5hi as a code owner October 8, 2023 03:12
@dasa
Copy link
Contributor

dasa commented Oct 9, 2023

A future enhancement could be to use os.availableParallelism (requires v18.14+), since it's recommended over using os.cpus().length.

@easrng easrng requested a review from shellscape as a code owner October 9, 2023 19:04
@shellscape
Copy link
Collaborator

Looks like you need an upstream update:

git remote add upstream [email protected]:rollup/plugins.git
git fetch upstream
git merge upstream/master
git push

That should get you set so that Windows Node 20 action passes.

@shellscape
Copy link
Collaborator

Alright I updated your fork for you from upstream/master and looks like there's a failure on Windows directly related to WASM. Please have a look.

@@ -356,3 +358,21 @@ test.serial('terser preserve vars in nameCache when provided', async (t) => {
}
});
});

test.serial('minify when os.cpus().length === 0', async (t) => {
const original = os.cpus;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe the test needs to also make os.availableParallelism unavailable, otherwise it won't use os.cpus.

@easrng
Copy link
Author

easrng commented Oct 16, 2023

Alright I updated your fork for you from upstream/master and looks like there's a failure on Windows directly related to WASM. Please have a look.

Aren't V8 errors either bugs in node itself or OOMs usually?

@shellscape
Copy link
Collaborator

Not sure in this case. It doesn't happen on the Ubuntu builds. The existing code doesn't throw this error, so it's possible the changes have surfaced a bug.

@shellscape shellscape force-pushed the master branch 3 times, most recently from c544fde to a929458 Compare October 25, 2023 19:15

test.serial('minify when os.cpus().length === 0', async (t) => {
const original = os.cpus;
os.cpus = () => [];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe overwriting os.cpus in this way is breaking Node 20.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@easrng could you please take a look?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@easrng should this also include os.availableParallelism = null;?

@shellscape shellscape force-pushed the master branch 4 times, most recently from a69c299 to a9b9cd3 Compare October 25, 2023 21:01
@dasa
Copy link
Contributor

dasa commented Dec 22, 2023

Maybe this change is worth a try?

diff --git a/packages/terser/test/test.js b/packages/terser/test/test.js
index 9dabacd..fd36d58 100644
--- a/packages/terser/test/test.js
+++ b/packages/terser/test/test.js
@@ -360,7 +360,9 @@ test.serial('terser preserve vars in nameCache when provided', async (t) => {
 });
 
 test.serial('minify when os.cpus().length === 0', async (t) => {
-  const original = os.cpus;
+  const originalAP = os.availableParallelism;
+  const originalCPUS = os.cpus;
+  os.availableParallelism = null;
   os.cpus = () => [];
   try {
     const bundle = await rollup({
@@ -373,6 +375,7 @@ test.serial('minify when os.cpus().length === 0', async (t) => {
     t.is(output.code, '"use strict";window.a=5,window.a<3&&console.log(4);\n');
     t.falsy(output.map);
   } finally {
-    os.cpus = original;
+    os.availableParallelism = originalAP;
+    os.cpus = originalCPUS;
   }
 });

It passes locally on Node 21.5.0.

@dasa
Copy link
Contributor

dasa commented Jul 30, 2024

@shellscape I can't tell if the tests ran after @easrng last commit. If not, would it be possible to run them again?

@shellscape
Copy link
Collaborator

@easrng @dasa there are some conflicts that will need to be resolved before I can force the tests to run again. you can fix the pnpm-lock conflict by running pnpm i in the repo root.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Error: Unexpected early exit
4 participants