Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

intermittent issue in wallet_upgradewallet.py: AssertionError: bdb magic does not match bdb btree magic #31210

Open
fanquake opened this issue Nov 4, 2024 · 6 comments · May be fixed by #30125
Open

intermittent issue in wallet_upgradewallet.py: AssertionError: bdb magic does not match bdb btree magic #31210

fanquake opened this issue Nov 4, 2024 · 6 comments · May be fixed by #30125
Labels
CI failed

Comments

@fanquake
Copy link
Member

fanquake commented Nov 4, 2024

https://cirrus-ci.com/task/5232872305459200:

067]  node0 2024-11-04T11:58:48.064759Z [httpworker.0] [src/wallet/wallet.h:936] [WalletLogPrintf] [default wallet] m_address_book.size() = 2 
[06:58:49.067]  test  2024-11-04T11:58:48.065000Z TestFramework (INFO): Can upgrade to HD 
[06:58:49.067]  test  2024-11-04T11:58:48.066000Z TestFramework (ERROR): Assertion failed 
[06:58:49.067]                                    Traceback (most recent call last):
[06:58:49.067]                                      File "/ci_container_base/test/functional/test_framework/test_framework.py", line 132, in main
[06:58:49.067]                                        self.run_test()
[06:58:49.067]                                      File "/ci_container_base/ci/scratch/build-x86_64-pc-linux-gnu/test/functional/wallet_upgradewallet.py", line 226, in run_test
[06:58:49.067]                                        orig_kvs = dump_bdb_kv(node_master_wallet)
[06:58:49.067]                                      File "/ci_container_base/test/functional/test_framework/bdb.py", line 142, in dump_bdb_kv
[06:58:49.067]                                        dump_meta_page(pages[INNER_META_PAGE])
[06:58:49.067]                                      File "/ci_container_base/test/functional/test_framework/bdb.py", line 100, in dump_meta_page
[06:58:49.067]                                        assert magic == BTREE_MAGIC, 'bdb magic does not match bdb btree magic'
[06:58:49.067]                                    AssertionError: bdb magic does not match bdb btree magic
[06:58:49.067]  test  2024-11-04T11:58:48.067000Z TestFramework (DEBUG): Closing down network thread

Looking at all the logs in #30798, this seems to be a different issue.
@maflcko
Copy link
Member

maflcko commented Nov 4, 2024

This only reproduces on a specific machine. I'll try to take a closer look.

@maflcko
Copy link
Member

maflcko commented Nov 4, 2024 •
edited
Loading

Steps to reproduce on a fresh Ubuntu 24.04 podman container on that machine:

export DEBIAN_FRONTEND=noninteractive && apt update && apt install curl wget htop git vim ccache -y && git clone https://github.com/bitcoin/bitcoin.git --depth=1 ./b-c && cd b-c && apt install build-essential cmake pkg-config libevent-dev libboost-dev libsqlite3-dev libdb++-dev python3 -y && cmake -B ./bld-cmake -DBUILD_TESTS=OFF -DWITH_BDB=ON -DENABLE_WALLET=ON -DBUILD_GUI=OFF -DBUILD_FUZZ_BINARY=OFF -DBUILD_BENCH=OFF -DWITH_ZMQ=OFF && cmake --build ./bld-cmake --parallel $(nproc) && test/get_previous_releases.py -b && ./bld-cmake/test/functional/wallet_upgradewallet.py --legacy-wallet

@maflcko
Copy link
Member

maflcko commented Nov 4, 2024

Possibly related: containers/podman#23808

I'll try tomorrow without a container.

@maflcko
Copy link
Member

maflcko commented Nov 5, 2024

Not sure how to fix this. IIUC neither podman (containers/podman#11415 (comment)), nor docker (https://docs.docker.com/engine/security/rootless/#known-limitations) support rootless zfs.

@achow101
Copy link
Member

achow101 commented Nov 5, 2024

I don't think this issue has to do with ZFS. I'm debugging the test on the problematic system and it seems that BDB decided to use a page size of 16384 for some reason, rather than our expected and hardcoded 4096. Changing it to 16384 resolves this, but we should instead be using the pagesize given by the wallet file rather than a fixed pagesize. This is a test only issue as the C++ parser does not use a fixed pagesize.

@achow101 achow101 mentioned this issue Nov 5, 2024
Closed
@maflcko
Copy link
Member

maflcko commented Nov 6, 2024

Thanks, tested that #31222 (74ff846) or #30125 (d45eb39) fix the issue.

I reproduced in a fresh install of Ubuntu 24.10 VM with zfs (experimental) selected, then ran the test in podman.

@fanquake fanquake linked a pull request Nov 7, 2024 that will close this issue
test: improve BDB parser (handle internal/overflow pages, support all page sizes) #30125
Open
@maflcko maflcko mentioned this issue Nov 20, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
CI failed
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

test: improve BDB parser (handle internal/overflow pages, support all page sizes) theStack/bitcoin
tests: Handle BDB dynamic pagesize achow101/bitcoin
3 participants
@fanquake @achow101 @maflcko