Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bitcoin 0.17 locking PRs #5034

Open
wants to merge 16 commits into
base: master
Choose a base branch
from

Conversation

LarryRuane
Copy link
Collaborator

@LarryRuane LarryRuane commented Mar 23, 2021

These are locking changes from upstream (bitcoin core) release 0.17 (Feb 16, 2018 to Aug 13, 2018), oldest to newest (when merged to the master branch).

This PR is complete and ready for review.

Each commit also includes a reference both to the PR and the upstream commit.

PRs and when each was merged to master:

@LarryRuane LarryRuane added C-upstream-port Category: Changes that are ported from the Bitcoin Core codebase. I-race Problems and improvements related to race conditions. labels Mar 23, 2021
@LarryRuane LarryRuane requested review from nuttycom, daira and str4d March 23, 2021 00:23
@LarryRuane LarryRuane self-assigned this Mar 23, 2021
@mdr0id mdr0id added the safe-to-build Used to send PR to prod CI environment label Mar 31, 2021
@zkbot
Copy link
Contributor

zkbot commented Apr 2, 2021

☔ The latest upstream changes (presumably #5009) made this pull request unmergeable. Please resolve the merge conflicts.

@LarryRuane
Copy link
Collaborator Author

Rebased on master, now that this PR is next in line. This is ready for review and possibly merging.

@LarryRuane LarryRuane marked this pull request as draft April 28, 2021 14:57
@LarryRuane LarryRuane marked this pull request as ready for review April 28, 2021 21:30
daira
daira previously approved these changes May 18, 2021
Copy link
Contributor

@daira daira left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, but @str4d needs to do a more careful review for interactions with other backported/Zcash-specific changes.

@r3ld3v r3ld3v added the S-committed Status: Planned work in a sprint label May 24, 2021
Copy link
Contributor

@str4d str4d left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One required change, and one mis-cherry-pick I'd like fixed (to make bitcoin/bitcoin@ecc3c4a a bit easier to review).

src/net.cpp Outdated Show resolved Hide resolved
src/txmempool.h Show resolved Hide resolved
src/main.cpp Show resolved Hide resolved
src/main.cpp Outdated Show resolved Hide resolved
src/main.cpp Outdated Show resolved Hide resolved
@LarryRuane LarryRuane force-pushed the upstream-locking-0.17 branch from b03729d to b23bc94 Compare May 27, 2021 06:31
@LarryRuane
Copy link
Collaborator Author

Thanks, @str4d, I think I've resolved all your comments. Regarding #5034 (comment), I went ahead and added bitcoin/bitcoin@fb8fad1 to the set.

@str4d str4d removed the S-committed Status: Planned work in a sprint label Jun 10, 2021
@str4d str4d removed this from the Core Sprint 2021-22 milestone Jun 10, 2021
promag and others added 15 commits January 13, 2023 16:55
When accessing mapBlockIndex cs_main must be held.

zcash: cherry picked from commit bitcoin/bitcoin@f814a3e
zcash: bitcoin/bitcoin#11041
FlushStateToDisk(...) won't do anything besides check if we need to prune if
FLUSH_STATE_NONE is given. We avoid reading the variable fCheckForPruning
which is guarded by the mutex cs_LastBlockFile.

zcash: cherry picked from commit bitcoin/bitcoin@2311c7c
zcash: bitcoin/bitcoin#11617
Ensures that callbacks are invoked in the order in which the chain is updated
Resolves #12978

(cherry picked from commit bitcoin/bitcoin@bitcoin/bitcoin@d86edd3)
* reading variable 'mapTx' requires holding mutex 'cs'
* reading variable 'mapNextTx' requires holding mutex 'cs'
* reading variable 'nCheckFrequency' requires holding mutex 'cs'

zcash: cherry picked from commit bitcoin/bitcoin@6bc5b71
zcash: bitcoin/bitcoin#11689
* writing variable 'nCheckFrequency' requires holding mutex 'cs'

zcash: cherry picked from commit bitcoin/bitcoin@0e2dfa8
zcash: bitcoin/bitcoin#11689
Technically, some internal datastructures may be in an inconsistent
state if we do this, though there are no known bugs there. Still,
for future safety, its much better to only unlock cs_main if we've
made progress (not just tried a reorg which may make progress).

zcash: cherry picked from commit bitcoin/bitcoin@ecc3c4a
zcash: bitcoin/bitcoin#13023

Zcash: Only includes part of the comment change to UpdatedBlockTip(), as
we don't have BlockConnected() yet.
If multiple threads are invoking ActivateBestChain, it was possible to have
them working towards different tips, and we could arrive at a less work tip
than we should.  Fix this by introducing a ChainState lock which must
be held for the entire duration of ActivateBestChain to enforce
exclusion in ABC.

zcash: cherry picked from commit bitcoin/bitcoin@a3ae8e6
zcash: bitcoin/bitcoin#13023
@str4d
Copy link
Contributor

str4d commented Jan 13, 2023

Rebased on master and addressed my two previous comments. Running CI to see what the test failure is.

@str4d str4d added the safe-to-build Used to send PR to prod CI environment label Jan 13, 2023
@ECC-CI ECC-CI removed the safe-to-build Used to send PR to prod CI environment label Jan 13, 2023
@str4d str4d modified the milestones: Release 5.4.0, Release 5.5.0 Jan 19, 2023
@str4d
Copy link
Contributor

str4d commented Jan 19, 2023

The assertion failure is being caused by the defensive assertion added to CChain::FindFork in #5862. Now I need to figure out why the changes in this PR (which ostensibly are improving locking) are causing a potential null pointer dereference.

@str4d
Copy link
Contributor

str4d commented Jan 19, 2023

5cf6ebd is the problematic commit. This likely means that it is backported incorrectly (which is reasonable, it's a complex backport and our code isn't in the same state as the usptream code was).

@daira daira modified the milestones: Release 5.5.0, Release 5.6.0 Feb 13, 2023
@str4d str4d removed this from the Release 5.6.0 milestone May 18, 2023
@nuttycom nuttycom added this to the Release 5.7.0 milestone Jun 7, 2023
@str4d str4d modified the milestones: Release 5.7.0, Post 5.7.0 Sep 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-upstream-port Category: Changes that are ported from the Bitcoin Core codebase. I-race Problems and improvements related to race conditions.
Projects
None yet
Development

Successfully merging this pull request may close these issues.