Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up all known races/platform-specific UB at the time PR was opened #9708

Merged
merged 10 commits into from
Feb 11, 2017

Conversation

TheBlueMatt
Copy link
Contributor

Because we (finally) have C++11, there is no excuse for using ints/flags/anything concurrently.

#9695 (which this is based on) got most of them, but the last two (addrName and addrLocal) were left out because they might need more discussion. Here I opted for giving them a wrapper which does locked access, though that does mean creating copies of them during use.

@gmaxwell
Copy link
Contributor

gmaxwell commented Feb 7, 2017

Concept ACK. Lightly tested ack. Getting these cleaned up makes it much easier to use tools that catch serious bugs.

Copy link
Contributor

@dcousens dcousens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

light utACK 74b46ed

Copy link
Contributor

@gmaxwell gmaxwell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK

@laanwj laanwj added this to the 0.14.0 milestone Feb 9, 2017
src/net.cpp Outdated

void CNode::MaybeSetAddrName(const std::string& addrNameIn) {
LOCK(cs_addrName);
if (addrName == "") {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: .empty() please

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

@TheBlueMatt TheBlueMatt force-pushed the 2017-02-fix-copystats-races branch from 74b46ed to a51ecf7 Compare February 9, 2017 22:16
@theuni
Copy link
Member

theuni commented Feb 10, 2017

I added clang annotations for everything in net (I think) that's guarded by a mutex, and built with -Wthread-safety: theuni@df50d4f

This reveals that nearly all is good, with these exceptions: theuni@d185ca0

The only one that we really need to worry about for 0.14 is the cs_filter locking. However, because cs_filter is locked from net_processing with a wide scope in a few places, I'm not confident that the LOCK(cs_filter) in AttemptToEvictConnection() can't deadlock due to ordering.

So, options are:

  • Ignore it for now, the concurrent access isn't new
  • pfilter -> atomic_pointer, fRelayTxes -> atomic_bool
  • Something like theuni@c6a93bd

I'm fine with any of the above.

@TheBlueMatt TheBlueMatt changed the title Clean Up all known races/platform-specific UB Clean up all known races/platform-specific UB at the time PR was opened Feb 10, 2017
@theuni
Copy link
Member

theuni commented Feb 10, 2017

Heh, title change comes from IRC discussion. The race mentioned above is not new, and not likely to cause issues. It's not worth holding up 0.14 for another set of ACKs or another PR to review.

If anyone strongly disagrees I'll PR a fix, but I'm ok with leaving it alone.

ACK a51ecf7ce00096bf607d15227ff1e1e39a3f6803

@laanwj
Copy link
Member

laanwj commented Feb 10, 2017

What should hold up the release are critical issues, e.g. those that can (or do) cause crashes, and corruption. Fixing every little thing static and dynamic analyzers can complain about is not part of that, and can be done later.

utACK a51ecf7

theuni and others added 10 commits February 10, 2017 11:32
These are (afaik) all long-standing races or concurrent accesses. Going
forward, we can clean these up so that they're not all individual atomic
accesses.

- Reintroduce cs_vRecv to guard receive-specific vars
- Lock vRecv/vSend for CNodeStats
- Make some vars atomic.
- Only set the connection time in CNode's constructor so that it doesn't change
@TheBlueMatt TheBlueMatt force-pushed the 2017-02-fix-copystats-races branch from a51ecf7 to db2dc7a Compare February 10, 2017 16:32
@TheBlueMatt
Copy link
Contributor Author

Rebased to fix a trivial merge conflict.

@theuni
Copy link
Member

theuni commented Feb 10, 2017

re-ACK db2dc7a

@sipa
Copy link
Member

sipa commented Feb 10, 2017

ACK db2dc7a. Running a node with master + this PR, with many connections, compiled with -fsanitize=thread -fsanitize=undefined. Only these cases are detected:

  • BDB's internal locks (which persist across call stacks, bleh) trigger lock inversion detections.
  • leveldb::port::AtomicPointer isn't recognized as an atomic (and should probably be replaced with c++11 std::atomic<T*>).
  • A suprious race is detected in httpserver.cpp's WorkQueue constructor.

@sipa sipa merged commit db2dc7a into bitcoin:master Feb 11, 2017
sipa added a commit that referenced this pull request Feb 11, 2017
…e PR was opened

db2dc7a Move CNode::addrLocal access behind locked accessors (Matt Corallo)
036073b Move CNode::addrName accesses behind locked accessors (Matt Corallo)
d8f2b8a Make nTimeBestReceived atomic (Matt Corallo)
22b4966 Move [clean|str]SubVer writes/copyStats into a lock (Matt Corallo)
0f31872 Make nServices atomic (Matt Corallo)
96f42d8 Make nStartingHeight atomic (Matt Corallo)
512731b Access fRelayTxes with cs_filter lock in copyStats (Matt Corallo)
ae683c1 Avoid copying CNodeStats to make helgrind OK with buggy std::string (Matt Corallo)
644f123 Make nTimeConnected const in CNode (Matt Corallo)
321d0fc net: fix a few races. Credit @TheBlueMatt (Cory Fields)
LarryRuane pushed a commit to LarryRuane/zcash that referenced this pull request Apr 1, 2021
These are (afaik) all long-standing races or concurrent accesses. Going
forward, we can clean these up so that they're not all individual atomic
accesses.

- Reintroduce cs_vRecv to guard receive-specific vars
- Lock vRecv/vSend for CNodeStats
- Make some vars atomic.
- Only set the connection time in CNode's constructor so that it doesn't change

zcash: cherry picked from commit 321d0fc
zcash: bitcoin/bitcoin#9708
LarryRuane pushed a commit to LarryRuane/zcash that referenced this pull request Apr 1, 2021
zcash: cherry picked from commit 644f123
zcash: bitcoin/bitcoin#9708
LarryRuane pushed a commit to LarryRuane/zcash that referenced this pull request Apr 1, 2021
str4d pushed a commit to LarryRuane/zcash that referenced this pull request Apr 1, 2021
str4d pushed a commit to LarryRuane/zcash that referenced this pull request Apr 1, 2021
zcash: cherry picked from commit 96f42d8
zcash: bitcoin/bitcoin#9708
str4d pushed a commit to LarryRuane/zcash that referenced this pull request Apr 1, 2021
zcash: cherry picked from commit 0f31872
zcash: bitcoin/bitcoin#9708
str4d pushed a commit to LarryRuane/zcash that referenced this pull request Apr 1, 2021
str4d pushed a commit to LarryRuane/zcash that referenced this pull request Apr 1, 2021
zcash: cherry picked from commit d8f2b8a
zcash: bitcoin/bitcoin#9708
str4d pushed a commit to LarryRuane/zcash that referenced this pull request Apr 1, 2021
str4d pushed a commit to LarryRuane/zcash that referenced this pull request Apr 1, 2021
zkbot added a commit to zcash/zcash that referenced this pull request Apr 2, 2021
Bitcoin 0.14 locking PRs

These are locking changes from upstream (bitcoin core) release 0.14, oldest to newest (when merged to the master branch).

Each commit also includes a reference both to the PR and the upstream commit.

- bitcoin/bitcoin#8472
- bitcoin/bitcoin#8606
  - Excludes a lock move because we don't have bitcoin/bitcoin#7840 which this move was partially-reverting.
- bitcoin/bitcoin#9230
  - Only first commit (we don't have `LogTimestampStr` anymore).
- bitcoin/bitcoin#9243
  - Only the sixth commit, excluding `IsArgSet` locking (we haven't pulled that function in yet).
- bitcoin/bitcoin#9626
  - The cherry-picked commit does not match the upstream at all, but the resulting lock is useful.
- bitcoin/bitcoin#9679
- bitcoin/bitcoin#9227
- bitcoin/bitcoin#9698
  - Excludes change to `CConnman::PushMessage` in second commit (which we don't have yet).
- bitcoin/bitcoin#9708
- bitcoin/bitcoin#9771
LarryRuane pushed a commit to LarryRuane/zcash that referenced this pull request Apr 26, 2021
LarryRuane pushed a commit to LarryRuane/zcash that referenced this pull request Jun 1, 2021
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants