General summary

I'm testing out upgrading from 2.13.1 to 2.15.5 for our company monorepo. I've found that there has been a serious performance drop for stack build in the newest version. Though, I noticed that 2.13.1 is also significantly slower than 2.7.5.

stack build has gotten slower when there's nothing to build with many packages. If one runs stack build followed immediately by another stack build (no-op), we see that the no-op build takes a significant amount of time, when I would expect it to be pretty fast (ideally <2 s).

We have >80 package.yamls in our repo. When I do a verbose build, I've noticed that after the Constructing the build plan log line, things go slowly. It seems that there's a sqlite database query for every package that's being run sequentially or something like that, and the time accumulates. There's references to getPackageFiles taking some amount of time. I assume the SQL queries are related to getPackageFiles in some way.

Here's a table showing my computer's run-time of the no-op stack build for various stack versions:

stack-2.7.5 is pretty reasonable, 2 s is totally acceptable for a big repo like ours. We're on 2.13.1 now clocking in at around 7 s, which is still annoying but workable. 22 s for a no-op build is pretty brutal.

Steps to reproduce

For example:

1. Run stack build
2. Run stack build again (should be a no-op)

Expected vs Actual

I'd expect the second stack build to be nearly instantaneous, but it's actually quite slow (see table above).

I've attached a gist of some censored debug output: https://gist.github.com/wraithm/86968037f99e8a10bf46e47f7d9abe6f

This gist has two files generated with:

stack-<stack version> --verbose build 2> >(rg getPackageFiles | sed -e 's/getPackageFiles.*/getPackageFiles/' > getPackageFiles-<stack version>.log)

where <stack version> is either 2.15.5 or 2.13.1, which corresponds to builds with those versions of stack.

You can see that the 2.15.5 one took about 22 s and the 2.13.1 one took about 7 s. In between those getPackageFiles log lines are SQL queries.

The SQL queries in the log look like this (again censored):

2024-04-05 15:19:14.413617: [debug] (SQL) SELECT "id","directory","type","pkg_src","active","path_env_var","haddock" FROM "config_cache" WHERE "directory"=? AND "type"=?; [PersistText "/path/to/repo/.stack-work/install/x86_64-linux-tinfo6/2ed63679412a6629a520726d957ec790c8a864b0615bc44b20bcbbcb34ba4597/9.4.8/",PersistText "lib:library-name"]
2024-04-05 15:19:14.413764: [debug] (SQL) SELECT "id", "config_cache_id", "index", "option" FROM "config_cache_dir_option" WHERE ("config_cache_id"=?) ORDER BY "index"; [PersistInt64 501]
2024-04-05 15:19:14.413916: [debug] (SQL) SELECT "id", "config_cache_id", "index", "option" FROM "config_cache_no_dir_option" WHERE ("config_cache_id"=?) ORDER BY "index"; [PersistInt64 501]
2024-04-05 15:19:14.414118: [debug] (SQL) SELECT "id", "config_cache_id", "ghc_pkg_id" FROM "config_cache_dep" WHERE ("config_cache_id"=?); [PersistInt64 501]
2024-04-05 15:19:14.414250: [debug] (SQL) SELECT "id", "config_cache_id", "component" FROM "config_cache_component" WHERE ("config_cache_id"=?); [PersistInt64 501]
2024-04-05 15:19:14.414528: [debug] Start: getPackageFiles /path/to/repo/path/to/packagename/packagename.cabal
2024-04-05 15:19:14.416066: [debug] Finished in 1ms: getPackageFiles /path/to/repo/path/to/packagename/packagename.cabal

Here's our (abbreviated) stack.yaml:

resolver: lts-21.22

flags:
  zlib:
    pkg-config: true
  xlsx:
    microlens: true

ghc-options:
  "$locals": -Werror
  "$everything": -haddock

Stack version

I'm mostly testing ghcup's stack-2.15.5:

$ stack --version
Version 2.15.5, Git revision 5649cc6b2522f51f0fc5543154b0fff868f9af31 x86_64 hpack-0.36.0

Method of installation

• Official binary, downloaded via haskellstack.org or from Stack's repository
• Via GHCup
• Via Cabal (the tool)
• An unofficial package repository (please specify which)
• Other (please specify)

I've tried both ghcup and nixpkgs for most of the versions in the table there, but the numbers all work out to be the same regardless of nixpkgs vs ghcup.

Platform

Your platform (machine architecture and operating system)

Intel x86_64 - Ubuntu 22.04