wallet: Keep track of the wallet's own transaction outputs in memory #27286
Conversation
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. Code Coverage & BenchmarksFor details see: https://corecheck.dev/bitcoin/bitcoin/pulls/27286. ReviewsSee the guideline for information on the review process.
If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
Need to load txs last so that IsMine works.
When loading transactions to the wallet, check the outputs, and keep track of the ones that are IsMine.
Since we track the outputs owned by the wallet with m_txos, we can now calculate the balance of the wallet by iterating m_txos and summing up the amounts of the unspent txos. As ISMINE_USED is not an actual isminetype that we attach to outputs and was just passed into `CachedTxGetAvailableCredit` for convenience, we pull the same determining logic from that function into `GetBalances` in order to preserve existing behavior.
Instead of iterating every transaction and every output stored in wallet when trying to figure out what outputs can be spent, iterate the TXO set which should be a lot smaller.
Instead of searching mapWallet for the preselected inputs, search m_txos. wallet_fundrawtransaction.py spends external inputs and needs the change output to also belong to the test wallet for the oversized tx test.
When a legacy wallet has been migrated to contain descriptors, but before the transactions have been updated to match, we need to recompute the wallet TXOs so that the transaction update will work correctly.
Instead of looking up the previous tx in mapWallet, and then calling IsMine on the specified output, use the TXO set and its cached IsMine value.
These two functions are no longer used as GetBalances now uses the TXO set rather than per-tx cached balances
One of the main issues with AvailableCoins is its performance when txs have unrelated outputs, so update the benchmark to check the performance of that.
|
Concept ACK, if this patchset can meaningfully improve performance and reliably refresh wallet utxos to keep the in-memory cache in sync. |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
|
I plan to review this soon. |
| @@ -4621,4 +4627,40 @@ std::optional<CKey> CWallet::GetKey(const CKeyID& keyid) const | |||
| } | |||
| return std::nullopt; | |||
| } | |||
|
|
|||
| void CWallet::RefreshSingleTxTXOs(const CWalletTx& wtx) | |||
There was a problem hiding this comment.
This method name looks funky. SingleTxTXO is short for single transaction transaction output?
Currently, the wallet is not actually aware about its own transaction outputs. Instead, it will iterate all of the transactions stored in
mapWallet, and then all of the outputs of those transactions, in order to figure out what belongs to it for the purposes of coin selection and balance calculation. For balance calculation, there is caching that results in it only iterating all of the transactions, but not all of the outputs. However when the cache is dirty, everything is iterated. This is especially problematic for wallets that have a lot of transactions, or transactions that have a lot of unrelated outputs (as may occur with coinjoins or batched payments).This PR helps to resolve this issue by making the wallet track all of the outputs that belong to it in a new member
m_txos. Note that this includes outputs that may have already been spent. Both balance calculation (GetBalance) and coin selection (AvailableCoins) are updated to iteratem_txos. This is generally faster since it ignores all of the unrelated outputs, and it is not slower as in the worst case of wallets containing only single output transactions, it's exactly the same number of outputs iterated.m_txosis memory only, and it is populated during wallet loading. When each transaction is loaded, all of its outputs are checked to see if it isIsMine, and if so, an entry added tom_txos. When new transactions are received, the same procedure is done.Since imports can change the
IsMinestatus of a transaction (although they can only be "promoted" from watchonly to spendable), all of the import RPCs will be a bit slower as they re-iterate all transactions and all outputs to updatem_txos.Each output in
m_txosis stored in a newWalletTXOclass. This class contains references to the parentCWalletTxand theCTxOutitself. It also caches theIsMinevalue of the txout. This should be safe asIsMineshould not change unless there are imports. This allows us to have additional performance improvements in places that use theseWalletTXOs as they can use the cachedIsMinerather than repeatedly callingIsMinewhich can be expensive.The existing
WalletBalancebenchmark demonstrates the performance improvement that this PR makes. The existingWalletAvailableCoinsbenchmark doesn't as all of the outputs used in that benchmark belong to the test wallet. I've updated that benchmark to have a bunch of unrelated outputs in each transaction so that the difference is demonstrated.This is part of a larger project to have the wallet actually track and store a set of its UTXOs.
Built on #24914 as it requires loading the txs last in order for
m_txosto be built correctly.Benchmarks:
Master:
WalletAvailableCoinsWalletBalanceCleanWalletBalanceDirtyWalletBalanceMineWalletBalanceWatchWalletCreateTxUseOnlyPresetInputsWalletCreateTxUsePresetInputsAndCoinSelectionWalletLoadingDescriptorsWalletLoadingLegacyPR:
WalletAvailableCoinsWalletBalanceCleanWalletBalanceDirtyWalletBalanceMineWalletBalanceWatchWalletCreateTxUseOnlyPresetInputsWalletCreateTxUsePresetInputsAndCoinSelectionWalletLoadingDescriptorsWalletLoadingLegacy