perf(compile): code cache #26528

dsherret · 2024-10-24T20:31:05Z

Adds a lazily created code cache to deno compile by default.

The code cache is created on first run to a single file in the temp directory and is only written once. After it's been written, the code cache becomes read only on subsequent runs. Only the modules loaded during startup are cached (dynamic imports are not code cached).

The code cache can be disabled by compiling with --no-code-cache.

> hyperfine --warmup 2 scratch.exe scratch_new.exe
Benchmark 1: scratch.exe
  Time (mean ± σ):     282.9 ms ±   6.5 ms    [User: 78.1 ms, System: 15.6 ms]
  Range (min … max):   276.5 ms … 295.2 ms    10 runs

Benchmark 2: scratch_new.exe
  Time (mean ± σ):     129.7 ms ±   6.8 ms    [User: 17.1 ms, System: 9.7 ms]
  Range (min … max):   120.9 ms … 138.4 ms    21 runs

Summary
  scratch_new.exe ran
    2.18 ± 0.13 times faster than scratch.exe

Closes #26502

felipecrs · 2024-10-29T13:03:22Z

This is very promising. Have you thought of including the initial code cache inside the compiled executable as well?

dsherret · 2024-11-04T18:17:45Z

Yeah, maybe at a later time and as a separate option. The problem is it will make the binary much larger.

felipecrs · 2024-11-04T18:28:17Z

It all depends on the use case. For some use cases, improving the startup time on the second attempt onwards as opposed to in the first attempt in favor of a reduced binary size will make a lot of sense.

For other cases, it will not make any sense at all. For example: containers. There's only one startup attempt in that case.

bartlomieju

LGTM

bartlomieju · 2024-11-18T17:44:22Z

cli/standalone/mod.rs

@@ -337,6 +380,23 @@ impl ModuleLoader for EmbeddedModuleLoader {
      ))),
    }
  }
+
+  fn code_cache_ready(


Unrelated to this PR, we should make this method be able to return sync or async response to save some ticks maybe.

felipecrs · 2024-11-21T16:26:32Z

@dsherret, how do I verify code cache is actually working on Deno 2.1.0?

felipecrs · 2024-11-21T16:27:39Z

I ran a benchmark and the improvements I found are very subtle, I wonder if code cache is indeed working:

$ hyperfine --warmup 2 --shell none --runs 50 './pkgx-deno-2.0.6 [email protected] --version' './pkgx-deno-2.1.0 [email protected] --version'
Benchmark 1: ./pkgx-deno-2.0.6 [email protected] --version
  Time (mean ± σ):     560.8 ms ±  17.2 ms    [User: 580.2 ms, System: 216.7 ms]
  Range (min … max):   534.8 ms … 615.1 ms    50 runs
 
Benchmark 2: ./pkgx-deno-2.1.0 [email protected] --version
  Time (mean ± σ):     523.0 ms ±   8.3 ms    [User: 549.1 ms, System: 224.0 ms]
  Range (min … max):   508.6 ms … 549.8 ms    50 runs
 
Summary
  ./pkgx-deno-2.1.0 [email protected] --version ran
    1.07 ± 0.04 times faster than ./pkgx-deno-2.0.6 [email protected] --version

bartlomieju · 2024-11-21T16:33:48Z

@felipecrs if you compile with --log-level debug you should be able to see logs like Loaded N code cache entries

felipecrs · 2024-11-21T16:39:27Z

Thank you, I guess there's something wrong then:

❯ ./pkgx --version 2>&1 | grep cache
DEBUG RS - denort::cache::cache_db:170 - Opening cache /home/felipecrs/.cache/deno/node_analysis_cache_v2...
DEBUG RS - denort::standalone::code_cache:56 - Failed to deserialize code cache: Cache key mismatch

❯ ./pkgx --version 2>&1 | grep cache
DEBUG RS - denort::cache::cache_db:170 - Opening cache /home/felipecrs/.cache/deno/node_analysis_cache_v2...
DEBUG RS - denort::standalone::code_cache:56 - Failed to deserialize code cache: Cache key mismatch

❯ deno clean
Removed /home/felipecrs/.cache/deno (7865 files, 58.14MB)

❯ ./pkgx --version 2>&1 | grep cache
DEBUG RS - denort::cache::cache_db:170 - Opening cache /home/felipecrs/.cache/deno/node_analysis_cache_v2...
DEBUG RS - denort::standalone::code_cache:56 - Failed to deserialize code cache: Cache key mismatch
DEBUG RS - denort::cache::cache_db:389 - Created parent directory for cache db.

❯ ./pkgx --version 2>&1 | grep cache
DEBUG RS - denort::cache::cache_db:170 - Opening cache /home/felipecrs/.cache/deno/node_analysis_cache_v2...
DEBUG RS - denort::standalone::code_cache:56 - Failed to deserialize code cache: Cache key mismatch

Any ideas?

dsherret · 2024-11-21T16:41:14Z

@felipecrs can you open an issue with a reproduction?

felipecrs · 2024-11-21T16:46:27Z

@felipecrs can you open an issue with a reproduction?

Deno compile code cache seems not working when error thrown during module loading #26976

felipecrs · 2024-11-21T17:21:01Z

Ok, I just benchmarked against --no-code-cache:

❯ deno compile --lock=deno.lock --allow-all --output "$INIT_CWD/pkgx" ./entrypoint.ts
Check file:///home/felipecrs/repos/pkgx/entrypoint.ts
Compile file:///home/felipecrs/repos/pkgx/entrypoint.ts to /home/felipecrs/repos/pkgx/pkgx

❯ mv pkgx pkgx-cc

❯ deno compile --lock=deno.lock --allow-all --output "$INIT_CWD/pkgx" --no-code-cache ./entrypoint.ts
Compile file:///home/felipecrs/repos/pkgx/entrypoint.ts to /home/felipecrs/repos/pkgx/pkgx

❯ mv pkgx pkgx-no-cc

❯ hyperfine --warmup 2 --shell none --runs 50 './pkgx-no-cc [email protected] --version' './pkgx-cc [email protected] --version'
Benchmark 1: ./pkgx-no-cc [email protected] --version
  Time (mean ± σ):     550.1 ms ±  18.8 ms    [User: 575.0 ms, System: 219.5 ms]
  Range (min … max):   525.6 ms … 606.3 ms    50 runs
 
Benchmark 2: ./pkgx-cc [email protected] --version
  Time (mean ± σ):     532.3 ms ±  16.5 ms    [User: 560.5 ms, System: 212.0 ms]
  Range (min … max):   515.2 ms … 572.9 ms    50 runs
 
Summary
  ./pkgx-cc [email protected] --version ran
    1.03 ± 0.05 times faster than ./pkgx-no-cc [email protected] --version

And indeed, we have an improvement. It is not fair to say the improvement is subtle, because it depends entirely on the application being tested and how long it takes to start.

Thank you very much, this is a really nice addition to Deno.

dsherret · 2024-11-21T17:37:24Z

The performance improvement will be most noticable when there's files that are quite large. For a bunch of smallish files it won't be that much (in the example above I'm importing a large bundled file)

dsherret and others added 18 commits October 22, 2024 19:19

perf(compile): use less memory

7236756

working now with typescript

1928daf

Merge branch 'main' into perf_deno_compile_less_memory

32913e7

Tell v8 that something is a string like before.

788f4ab

fix byonm issue

742ae3f

Merge branch 'main' into perf_deno_compile_less_memory

1ad5f62

maybe fix ci

fd13339

do not store data urls in the binary

1e0dabd

switch to le because this is not network

d7cd10b

review

c156944

perf(compile): code cache for initial load

938c3e0

use distinct strategies for compile

0a7c050

use distinct strategies for compile

0957e09

tests

bad329a

support --no-code-cache

e9e4ad2

lint

8a910fd

Merge branch 'main' into perf_compile_code_cache

bc08455

remove unused use

815be3a

dsherret added this to the 2.1.0 milestone Oct 24, 2024

dsherret added 3 commits October 24, 2024 16:47

fix test

ec0d0c7

Do not subtract with overflow when deserializing.

1140105

maybe fix test failing because they had the same binary name

5b13157

bartlomieju mentioned this pull request Oct 29, 2024

can deno compile into bytecode? #8820

Closed

dsherret requested a review from bartlomieju November 18, 2024 17:36

dsherret added 3 commits November 18, 2024 13:11

Merge branch 'main' into perf_compile_code_cache

d360dda

update after merge

488f6dc

lint

0e38fab

bartlomieju approved these changes Nov 18, 2024

View reviewed changes

Merge branch 'main' into perf_compile_code_cache

b1a784b

dsherret enabled auto-merge (squash) November 18, 2024 20:03

dsherret merged commit dd4570e into denoland:main Nov 18, 2024
17 checks passed

dsherret deleted the perf_compile_code_cache branch November 18, 2024 20:31

This was referenced Nov 21, 2024

Upgrade Deno from v2.0.2 to v2.1.1 pkgxdev/pkgx#1049

Open

Deno compile code cache seems not working when error thrown during module loading #26976

Closed

felipecrs mentioned this pull request Nov 21, 2024

deno compile --include-code-cache #26979

Open

felipecrs mentioned this pull request Nov 21, 2024

deno compile --code-cache-path #26980

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

perf(compile): code cache #26528

perf(compile): code cache #26528

dsherret commented Oct 24, 2024 •

edited

Loading

felipecrs commented Oct 29, 2024

dsherret commented Nov 4, 2024

felipecrs commented Nov 4, 2024 •

edited

Loading

bartlomieju left a comment

bartlomieju Nov 18, 2024

felipecrs commented Nov 21, 2024

felipecrs commented Nov 21, 2024 •

edited

Loading

bartlomieju commented Nov 21, 2024

felipecrs commented Nov 21, 2024 •

edited

Loading

dsherret commented Nov 21, 2024

felipecrs commented Nov 21, 2024

felipecrs commented Nov 21, 2024 •

edited

Loading

dsherret commented Nov 21, 2024 •

edited

Loading

perf(compile): code cache #26528

perf(compile): code cache #26528

Conversation

dsherret commented Oct 24, 2024 • edited Loading

felipecrs commented Oct 29, 2024

dsherret commented Nov 4, 2024

felipecrs commented Nov 4, 2024 • edited Loading

bartlomieju left a comment

Choose a reason for hiding this comment

bartlomieju Nov 18, 2024

Choose a reason for hiding this comment

felipecrs commented Nov 21, 2024

felipecrs commented Nov 21, 2024 • edited Loading

bartlomieju commented Nov 21, 2024

felipecrs commented Nov 21, 2024 • edited Loading

dsherret commented Nov 21, 2024

felipecrs commented Nov 21, 2024

felipecrs commented Nov 21, 2024 • edited Loading

dsherret commented Nov 21, 2024 • edited Loading

dsherret commented Oct 24, 2024 •

edited

Loading

felipecrs commented Nov 4, 2024 •

edited

Loading

felipecrs commented Nov 21, 2024 •

edited

Loading

felipecrs commented Nov 21, 2024 •

edited

Loading

felipecrs commented Nov 21, 2024 •

edited

Loading

dsherret commented Nov 21, 2024 •

edited

Loading