perf: analyze cjs exports and emit typescript in parallel #23856
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dsherret
changed the title
dsherret
commented
May 17, 2024
| @@ -113,6 +117,7 @@ impl<TCjsCodeAnalyzer: CjsCodeAnalyzer> NodeCodeTranslator<TCjsCodeAnalyzer> { | |||
| reexports_to_handle.push_back((reexport, specifier.clone())); | |||
| } | |||
|
|
|||
| // todo(dsherret): we could run this analysis concurrently in a FuturesOrdered | |||
There was a problem hiding this comment.
This is a future perf improvement. Opened #23859
dsherret
commented
May 17, 2024
| &self, | ||
| graph: &ModuleGraph, | ||
| ) -> Result<(), AnyError> { | ||
| // todo(dsherret): we could do this concurrently |
There was a problem hiding this comment.
Another future perf improvement #23860
bartlomieju
approved these changes
May 17, 2024
| } | ||
| } | ||
|
|
||
| fn load_prepared_module_sync( |
There was a problem hiding this comment.
Is this temporary or will we always need both sync and async versions?
There was a problem hiding this comment.
Source map lookup is sync, so we'll probably need this unless we move the spawn blocking to a higher level (I investigated doing this just now, and the performance is equivalent, but it means that all our code in the worker needs to be Send/Sync, which is something I don't think we want to do)
Comment on lines
+780
to
+784
| deno_core::ModuleLoadResponse::Async( | ||
| async move { | ||
| inner | ||
| .load_inner( | ||
| &specifier, |
tests/integration/inspector_tests.rs
Outdated
There was a problem hiding this comment.
Since it's async these now happen out of order sometimes.
bartlomieju
pushed a commit
that referenced
this pull request
May 21, 2024
bartlomieju
pushed a commit
that referenced
this pull request
May 21, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Two main things we can do in parallel:
This makes the loader async to make doing this easier.
Deleting the node analysis cache between runs and importing some npm packages (this only makes the first run much faster due to the caches... the second run is the same speed as befroe):
Deleting the TypeScript emit and module analysis cache between runs and importing multiple copies of Dax:
Note that this is still parsing typescript files sequentially because it doesn't include (denoland/deno_graph#477). It is just emitting typescript files in parallel.