-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add ability for node to reject tx from mempool by number of tx inputs #2342
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some minor changes needed. Also needs a test.
src/init.cpp
Outdated
@@ -332,6 +332,7 @@ std::string HelpMessage(HelpMessageMode mode) | |||
strUsage += HelpMessageOpt("-dbcache=<n>", strprintf(_("Set database cache size in megabytes (%d to %d, default: %d)"), nMinDbCache, nMaxDbCache, nDefaultDbCache)); | |||
strUsage += HelpMessageOpt("-loadblock=<file>", _("Imports blocks from external blk000??.dat file") + " " + _("on startup")); | |||
strUsage += HelpMessageOpt("-maxorphantx=<n>", strprintf(_("Keep at most <n> unconnectable transactions in memory (default: %u)"), DEFAULT_MAX_ORPHAN_TRANSACTIONS)); | |||
strUsage += HelpMessageOpt("-mempooltxvinlimit=<n>", _("If set, limit the number of inputs to a transaction which this node will accept into the mempool for relay and mining")); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suggest "mempooltxinputlimit"; vin
is internal jargon.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, it's the number of transparent inputs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
src/init.cpp
Outdated
if (mapArgs.count("-mempooltxvinlimit")) { | ||
unsigned int limit = GetArg("-mempooltxvinlimit", UINT_MAX); | ||
if (limit == 0) { | ||
return InitError(_("Mempool transaction vin limit cannot be 0.")); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also change this if vin
is dropped from the option name.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
src/init.cpp
Outdated
@@ -1009,6 +1010,14 @@ bool AppInit2(boost::thread_group& threadGroup, CScheduler& scheduler) | |||
} | |||
#endif | |||
|
|||
if (mapArgs.count("-mempooltxvinlimit")) { | |||
unsigned int limit = GetArg("-mempooltxvinlimit", UINT_MAX); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe the default should be 40, but let's discuss that on #2343.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Default is now 0 which means no limit (code has changed in forced push)
src/init.cpp
Outdated
@@ -1009,6 +1010,14 @@ bool AppInit2(boost::thread_group& threadGroup, CScheduler& scheduler) | |||
} | |||
#endif | |||
|
|||
if (mapArgs.count("-mempooltxvinlimit")) { | |||
unsigned int limit = GetArg("-mempooltxvinlimit", UINT_MAX); | |||
if (limit == 0) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
int64_t limit = GetArg("-mempooltxvinlimit", std::numeric_limits<int64_t>::max());
if (limit < 0) {
return InitError(_("Mempool limit on transparent inputs to a transaction must be greater than 0."));
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Have adopted language.
src/main.cpp
Outdated
// Node operator can choose to reject tx by number of inputs | ||
if (mapArgs.count("-mempooltxvinlimit")) { | ||
unsigned int n = tx.vin.size(); | ||
unsigned int limit = GetArg("-mempooltxvinlimit", UINT_MAX); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
static_assert(std::numeric_limits<size_t>::max() >= std::numeric_limits<int64_t>::max(), "size_t too small");
size_t n = tx.vin.size();
size_t limit = (size_t) GetArg("-mempooltxvinlimit", std::numeric_limits<int64_t>::max());
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
Red flag: Nathan mistakenly thought that the Incident Response team had required this ticket to be prioritised for 1.0.9. In fact the opposite was true, the Incident Response team (me as Executive Coordinator) had decided that we would take steps to fix the problem in the short term explicitly under the assumption that this would not imply that this ticket would be prioritised for 1.0.9. |
607ef82
to
9c2a621
Compare
@daira Changes made. No test yet, is someone available to write that? |
src/init.cpp
Outdated
@@ -332,6 +332,7 @@ std::string HelpMessage(HelpMessageMode mode) | |||
strUsage += HelpMessageOpt("-dbcache=<n>", strprintf(_("Set database cache size in megabytes (%d to %d, default: %d)"), nMinDbCache, nMaxDbCache, nDefaultDbCache)); | |||
strUsage += HelpMessageOpt("-loadblock=<file>", _("Imports blocks from external blk000??.dat file") + " " + _("on startup")); | |||
strUsage += HelpMessageOpt("-maxorphantx=<n>", strprintf(_("Keep at most <n> unconnectable transactions in memory (default: %u)"), DEFAULT_MAX_ORPHAN_TRANSACTIONS)); | |||
strUsage += HelpMessageOpt("-mempooltxinputlimit=<n>", _("Set the maximum number of transparent inputs in a transaction which the mempool will accept (default: 0 = no limit applied)")); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Non-blocking: s/which/that/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Other than the double-negative in the error message, ACK on the changes since my last review. Still needs a test.
src/init.cpp
Outdated
if (mapArgs.count("-mempooltxinputlimit")) { | ||
int64_t limit = GetArg("-mempooltxinputlimit", 0); | ||
if (limit < 0) { | ||
return InitError(_("Mempool limit on transparent inputs to a transaction cannot not be negative")); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/cannot not/cannot/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
A kind proposal, if there would a comment in the code about the problems and the solution, it would be great. I have been following the discussion in chat and I think I'd agree that "limiting this is good", but necessarily the real reasonin and why some particular limit was chosen (even commenting in vein "this limit is an educated guess to quell most of the problems known at the time") and what would happen to the transactions if the check fails (transactions in limbo?). |
@veikkoeeva I'll answer that on #2343. |
Concept NACK: #2343 (comment) |
Considering for 1.0.10 release juxtaposed to related tickets. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
utACK on the patch, but I agree with daira's string requests and that this needs a test.
I'm also +1 on merging this in 1.0.10. I agree with @daira that we need something that users can leverage now, while we work on the longer-term improvements. I also think that the release notes should be explicit about how this is a short-term fix, and that it will be replaced (not extended) in future (likely with some metric that takes into account the entire mempool, not just individual transactions). |
based on the number of transparent inputs.
9c2a621
to
da6d939
Compare
src/gtest/test_mempool.cpp
Outdated
CValidationState state3; | ||
CTransaction tx3(mtx); | ||
EXPECT_FALSE(AcceptToMemoryPool(pool, state3, tx3, false, &missingInputs)); | ||
EXPECT_NE(state3.GetRejectReason(), "bad-txns-version-too-low"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why are all the error messages in the test related to 'bad-txns-version-too-low' ? I only see it fired here:
bool CheckTransactionWithoutProofVerification(const CTransaction& tx, CValidationState &state
)
{
// Basic checks that don't depend on any context
// Check transaction version
if (tx.nVersion < MIN_TX_VERSION) {
return state.DoS(100, error("CheckTransaction(): version too low"),
REJECT_INVALID, "bad-txns-version-too-low");
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct. I intentionally set tx.nVersion = 0
to trigger this error, as it's the first check that occurs after the -mempooltxinputlimit
check, and it means that I didn't have to mock out a heap of global state. Think of that error as "the transaction passes the tx input limit check". The actual check itself doesn't return a reason, which is why I check that the reject reason is not bad-txns-version-too-low
; we could possibly make things more explicit by checking instead that it is empty.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you copy the above explanation into the test? I take it we don't want to return state.DoS
which updates global CValidationState state
in init.cpp
, rather than what we do right now which is simply return false (like some of the other code paths in AcceptToMemoryPool).
mempool rejection.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
NACK
if (isfromtaddr_) { | ||
size_t limit = (size_t)GetArg("-mempooltxinputlimit", 0); | ||
if (limit > 0) { | ||
size_t n = t_inputs_.size(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At this point in the function, this is the list of all available inputs, not selected inputs; z_sendmany
therefore fails incorrectly. This check needs to be shifted to just below where t_inputs_
is re-used for storing the selected inputs. We also need to re-write this code (likely as part of #1360) to prevent this kind of bug.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ACK
Also reduce number of z_sendmany calls made so test runs quicker.
I'll review now. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The comment about the order of checking tx version and number of inputs needs a response, otherwise utACK.
// Create an obviously-invalid transaction | ||
// We intentionally set tx.nVersion = 0 to reliably trigger an error, as | ||
// it's the first check that occurs after the -mempooltxinputlimit check, | ||
// and it means that we don't have to mock out a lot of global state. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wait, input limiting occurs before the tx version check? How can we parse the transaction if it has an unknown version? The spec explicitly says you're not supposed to try to parse fields other than nVersion if the version is unrecognised. (This could lead to misreporting a transaction as being invalid for having too many inputs, when in fact that field at that position has changed its meaning entirely and the transaction should instead have been rejected for having an unknown version. Not critical, but it could lead to confusing log messages.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did wonder this myself. However:
- The version check is solely that
tx.nVersion >= MIN_TX_VERSION
, whereMIN_TX_VERSION = 1
, so any positive integer is already accepted. This is in fact one way in which transaction replay prevention has been proposed upstream: make the version negative. - Transaction parsing already happens before
AcceptToMemoryPool
is called, so the spec should still be satisfied:
bool static ProcessMessage(...)
{
...
else if (strCommand == "tx")
{
vector<uint256> vWorkQueue;
vector<uint256> vEraseQueue;
CTransaction tx;
vRecv >> tx;
CInv inv(MSG_TX, tx.GetHash());
pfrom->AddInventoryKnown(inv);
LOCK(cs_main);
bool fMissingInputs = false;
CValidationState state;
pfrom->setAskFor.erase(inv.hash);
mapAlreadyAskedFor.erase(inv);
if (!AlreadyHave(inv) && AcceptToMemoryPool(mempool, state, tx, true, &fMissingInputs))
{
...
}
...
}
...
}
bool ProcessMessages(...)
{
...
bool fRet = false;
try
{
fRet = ProcessMessage(pfrom, strCommand, vRecv, msg.nTime);
boost::this_thread::interruption_point();
}
catch (const std::ios_base::failure& e)
{
...
}
...
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A bystander note here, feel free to ignore, but could this reasoning be added as a comment to the code? I think it would greatly help others too and later when one is perhaps considering refactoring the code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the reminder @veikkoeeva - I've updated the first post in the conversation above, so when this is merged, it will also be in the commit message for the merge. I think anybody in the future who wants to refactor the code around mempooltxinputlimit
will have the information on hand to do so.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't see this in the first post.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@daira You don't see the text beginning "Implement short-term... " ? Also the PR number should be part of the zkbot commit message, so a developer can find their way to this conversation.
Actually I'll file another ticket about the parsing-of-unknown-tx-versions issue. That is not going to affect consensus so it can be addressed separately and does not need to block this PR. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Needs one more adjustment, which I will make now.
assert_equal(set(self.nodes[1].getrawmempool()), set()) | ||
|
||
# Should use two UTXOs and succeed | ||
spend_taddr_id2 = self.nodes[1].sendtoaddress(node0_taddr, spend_taddr_output + spend_taddr_output - Decimal('1')) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The first call was changed from sendtoaddress
to sendfrom
; this one should be too.
self.call_z_sendmany(node0_zaddr, node1_taddr, spend_taddr_amount - spend_taddr_output - spend_taddr_output - Decimal('0.0001')) # note amount less fees | ||
recipients.append({"address":self.nodes[1].getnewaddress(), "amount": spend_taddr_output}) | ||
recipients.append({"address":self.nodes[1].getnewaddress(), "amount": spend_taddr_output}) | ||
recipients.append({"address":self.nodes[1].getnewaddress(), "amount": spend_taddr_amount - spend_taddr_output - spend_taddr_output}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ACK; the first variant was left over from an earlier version where node 1 already had a balance, so I was trying to get everything into one t-addr, but couldn't make multiple TXOs in the same z_sendmany
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ACK
@zkbot r+ |
📌 Commit 6ea58d1 has been approved by |
Add ability for node to reject tx from mempool by number of tx inputs Implement short-term solution described in #2343 so that users can respond promptly to critical short-term problems caused by quadratic validation scaling, such as the getblocktemplate latency, block propagation latency, and mempool size inflation issues described in #2333.
Implement short-term solution described in #2343 so that users can respond promptly to critical short-term problems caused by quadratic validation scaling, such as the getblocktemplate latency, block propagation latency, and mempool size inflation issues described in #2333.