Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make nWalletDBUpdated atomic to avoid a potential race. #9227

Merged
merged 1 commit into from
Feb 6, 2017

Conversation

pstratem
Copy link
Contributor

Title

@luke-jr
Copy link
Member

luke-jr commented Nov 27, 2016

utACK

@TheBlueMatt
Copy link
Contributor

If you're gonna touch this, might as well fix it - will it interact negatively with multiwallet support as written? Also, should be easy to make it only exist in walletdb.cpp, I'd think?

@laanwj
Copy link
Member

laanwj commented Nov 28, 2016

will it interact negatively with multiwallet support as written?

Right - this should probably exist on the CWalletDB object instead of globally, otherwise multiple wallets will react on each other's changes.
(and then you may not even need an atomic because they will be holding their respective wallet lock)
This is not entirely trivial to fix because the 'wallet flush' thread (which IIRC uses this) is decidedly single-wallet. Maybe one of the multiwallet pulls addresses this though.

@jonasschnelli
Copy link
Contributor

Agree with @laanwj.
Also, #8776 would be a step towards a better – multiwallet safe – wallet flush.

@luke-jr
Copy link
Member

luke-jr commented Nov 28, 2016

multiwallet makes the flush thread go over all the wallets and flush them all when this is updated. AFAIK extraneous flushing is mostly harmless...?

@laanwj
Copy link
Member

laanwj commented Nov 28, 2016

multiwallet makes the flush thread go over all the wallets and flush them all when this is updated. AFAIK extraneous flushing is mostly harmless...?

It's would be a performance sink. Remember the thread is named badly. Our 'flushing' isn't really flushing, it is a full fledged database consolidation.

Edit: so the idea of having a single flush thread visit all wallets is good, but the write tracking should be per-wallet and not global.

@TheBlueMatt
Copy link
Contributor

This should probably also move to CScheduler to avoid having Yet Another Thread when we dont need it.

@gmaxwell
Copy link
Contributor

The db flush here doesn't really flush a wallet at all, it flushes the whole database environment, so I don't believe it can be made per-wallet.

@pstratem
Copy link
Contributor Author

Indeed this cannot be made per wallet, this is flushing the entire database environment.

@jonasschnelli
Copy link
Contributor

Right now we are flushing the DB environment (https://docs.oracle.com/cd/E17276_01/html/api_reference/C/envclose.html) with DBEnv->CloseDb(filename).
Is there a reason why we wouldn't want to use DB->Sync() (https://docs.oracle.com/cd/E17276_01/html/api_reference/C/dbsync.html)?

@laanwj
Copy link
Member

laanwj commented Nov 30, 2016

Is there a reason why we wouldn't want to use DB->Sync()
(https://docs.oracle.com/cd/E17276_01/html/api_reference/C/dbsync.html)?

The wallet code is set up to flush after every (important) write immediately, that's not what the thread is for.

What the flush thread does is not really flushing. The method you propose would be a less hassle solution to do flushing, but what the thread wants to do is consolidate the database files: When writing, BerkeleyDB writes to log files and merges these into the main database only incidentally. The whole circuitous exercise in the thread is to force this to happen, so that the wallet is one file.

(can we please rename this thread and add a comment somewhere, everyone reading this code is confused about this, and this has been the case for years)

@laanwj
Copy link
Member

laanwj commented Nov 30, 2016

The db flush here doesn't really flush a wallet at all, it flushes the whole database environment, so I don't believe it can be made per-wallet.

Doesn't every wallet get its own database environment? I guess not.

@gmaxwell
Copy link
Contributor

No-- well every database environment needs it's own database directory, cache overheads, etc. So I'm guessing that wouldn't be a great move.

@pstratem pstratem force-pushed the 2016-11-26-nwalletdbupdated-race branch 4 times, most recently from 391927b to f820ac2 Compare December 1, 2016 02:06
@@ -11,6 +11,7 @@
#include "wallet/db.h"
#include "key.h"

#include <atomic>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: move this to the .cpp.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit picked

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not it wasnt?

@pstratem pstratem force-pushed the 2016-11-26-nwalletdbupdated-race branch from f820ac2 to d63ff62 Compare February 3, 2017 18:46
@TheBlueMatt
Copy link
Contributor

Can this get an 0.14 tag, since its a simple bugfix?

@TheBlueMatt
Copy link
Contributor

utACK d63ff62

1 similar comment
@sipa
Copy link
Member

sipa commented Feb 4, 2017

utACK d63ff62

@sipa sipa added this to the 0.14.0 milestone Feb 4, 2017
@laanwj laanwj merged commit d63ff62 into bitcoin:master Feb 6, 2017
laanwj added a commit that referenced this pull request Feb 6, 2017
d63ff62 Make nWalletDBUpdated atomic to avoid a potential race. (Patrick Strateman)
codablock pushed a commit to codablock/dash that referenced this pull request Jan 23, 2018
… race.

d63ff62 Make nWalletDBUpdated atomic to avoid a potential race. (Patrick Strateman)
andvgal pushed a commit to energicryptocurrency/gen2-energi that referenced this pull request Jan 6, 2019
… race.

d63ff62 Make nWalletDBUpdated atomic to avoid a potential race. (Patrick Strateman)
CryptoCentric pushed a commit to absolute-community/absolute that referenced this pull request Feb 27, 2019
… race.

d63ff62 Make nWalletDBUpdated atomic to avoid a potential race. (Patrick Strateman)
random-zebra added a commit to PIVX-Project/PIVX that referenced this pull request Aug 23, 2020
…al race

5069899 Make nWalletDBUpdated atomic to avoid a potential race. (furszy)
f533e37 move ThreadFlushWalletDB declaration to walletdb.h (Philip Kaufmann)

Pull request description:

  Quick back ports, coming from dashpay#5983 and bitcoin#9227.

ACKs for top commit:
  random-zebra:
    utACK 5069899
  Fuzzbawls:
    utACK 5069899

Tree-SHA512: 81c6726619f16f53e524bb29e72966c58b069a2bfbc10b1fab89fd28569f9fc555fd0c888b049173e3bb2efa530e62c757e28a3dd11297a66ebd066f1f722fd4
LarryRuane pushed a commit to LarryRuane/zcash that referenced this pull request Apr 1, 2021
zkbot added a commit to zcash/zcash that referenced this pull request Apr 2, 2021
Bitcoin 0.14 locking PRs

These are locking changes from upstream (bitcoin core) release 0.14, oldest to newest (when merged to the master branch).

Each commit also includes a reference both to the PR and the upstream commit.

- bitcoin/bitcoin#8472
- bitcoin/bitcoin#8606
  - Excludes a lock move because we don't have bitcoin/bitcoin#7840 which this move was partially-reverting.
- bitcoin/bitcoin#9230
  - Only first commit (we don't have `LogTimestampStr` anymore).
- bitcoin/bitcoin#9243
  - Only the sixth commit, excluding `IsArgSet` locking (we haven't pulled that function in yet).
- bitcoin/bitcoin#9626
  - The cherry-picked commit does not match the upstream at all, but the resulting lock is useful.
- bitcoin/bitcoin#9679
- bitcoin/bitcoin#9227
- bitcoin/bitcoin#9698
  - Excludes change to `CConnman::PushMessage` in second commit (which we don't have yet).
- bitcoin/bitcoin#9708
- bitcoin/bitcoin#9771
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants