-
Notifications
You must be signed in to change notification settings - Fork 38.4k
Split CConnman #30988
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Split CConnman #30988
Conversation
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. Code Coverage & BenchmarksFor details see: https://corecheck.dev/bitcoin/bitcoin/pulls/30988. ReviewsSee the guideline for information on the review process.
If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. LLM Linter (✨ experimental)Possible typos and grammar issues:
No other typographic or grammatical errors that make the English invalid or incomprehensible were found in the added lines. drahtbot_id_5_m |
|
Done. |
|
Here's an initial sketch of making |
tdb3
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Concept ACK
|
Concept ACK This looks great and the API in the header looks easy, thanks. I'm in the process of cleaning up my HTTP branch for a pull request and then I can start reviewing this and rebasing on top. One element of libevent I'm not immediately seeing here is timed events. Really the only thing HTTP needs it for is |
I wonder why the existing scheduler can't be used for re-locking the wallet? I know there is #18488 and #14289, but the thread is already filled with random stuff such as |
|
@pinheadmz, I think that the functionality of "execute this code after some time", is not much related to the sockets handling and better be implemented at some higher level, not inside /**
* SockMan has completed send+recv for all nodes.
* Can be used to execute periodic tasks for all nodes.
* The implementation in SockMan does nothing.
*/
virtual void EventIOLoopCompletedForAllPeers();Edit: I guess |
They were coupled in `struct ListenSocket`, but the socket belongs to the lower level transport protocol, whereas the permissions are specific to the higher Bitcoin P2P protocol.
Now that `CConnman::ListenSocket` is a `struct` that contains only one member variable of type `std::shared_ptr<Sock>`, drop `ListenSocket` and use `shared_ptr` directly. Replace the vector of `ListenSocket` with a vector of `shared_ptr`.
Introduce a new low-level socket managing class `SockMan` and move the `CConnman::BindListenPort()` method to it.
It was copied verbatim from `CConnman::BindListenPort()` in the previous commit. Modernize its variables and style and log the error messages from the caller. Also categorize the informative messages to the "net" category because they are quite specific to the networking layer.
Move the `CConnman::AcceptConnection()` method to `SockMan` and split parts of it: * the flip-to-CJDNS part: to just after the `AcceptConnection()` call * the permissions part: at the start of `CreateNodeFromAcceptedSocket()`
Move `CConnman::GetNewNodeId()` to `SockMan::GetNewId()`. Avoid using the word "node" because that is too specific for `CConnman`.
CConnman-specific or in other words, Bitcoin P2P specific. Now the `ThreadI2PAcceptIncoming()` method is protocol agnostic and can be moved to `SockMan`.
Change `CConnman::m_nodes` from `std::vector<CNode*>` to `std::unordered_map<NodeId, CNode*>` because interaction between `CConnman` and `SockMan` is going to be based on `NodeId` and finding a node by its id would better be fast. Change `PeerManagerImpl::EvictExtraOutboundPeers()` to account for nodes no longer always being in order of id. The old code would have failed to update `next_youngest_peer` correctly if `CConnman::m_nodes` hadn't always had nodes in ascending order of id. During fuzzing make sure that we don't generate duplicate `CNode` ids. The easiest way to do that is to use sequential ids. As a nice side effect the existent search-by-id operations in `CConnman::AttemptToEvictConnection()`, `CConnman::DisconnectNode()` and `CConnman::ForNode()` now become `O(1)` (were `O(number of nodes)`), as well as the erase in `CConnman::DisconnectNodes()`.
Move the parts of `CConnman::GenerateWaitSockets()` that are specific to the Bitcoin-P2P protocol to dedicated methods: `ShouldTryToSend()` and `ShouldTryToRecv()`. This brings us one step closer to moving `GenerateWaitSockets()` to the protocol agnostic `SockMan` (which would call `ShouldTry...()` from `CConnman`).
…cketHandler() Move some parts of `CConnman::SocketHandlerConnected()` and `CConnman::ThreadSocketHandler()` that are specific to the Bitcoin-P2P protocol to dedicated methods: `EventIOLoopCompletedForOne(id)` and `EventIOLoopCompletedForAll()`. This brings us one step closer to moving `SocketHandlerConnected()` and `ThreadSocketHandler()` to the protocol agnostic `SockMan` (which would call `EventIOLoopCompleted...()` from `CConnman`).
Introduce 4 new methods for the interaction between `CConnman` and `SockMan`: * `EventReadyToSend()`: called when there is readiness to send and do the actual sending of data. * `EventGotData()`, `EventGotEOF()`, `EventGotPermanentReadError()`: called when the corresponing recv events occur. These methods contain logic that is specific to the Bitcoin-P2P protocol and move it away from `CConnman::SocketHandlerConnected()` which will become a protocol agnostic method of `SockMan`. Also, move the counting of sent bytes to `CConnman::SocketSendData()` - both callers of that method called `RecordBytesSent()` just after the call, so move it from the callers to inside `CConnman::SocketSendData()`.
Move the protocol agnostic parts of `CConnman::ConnectNode()` into `SockMan::ConnectAndMakeId()` and leave the Bitcoin-P2P specific stuff in `CConnman::ConnectNode()`. Move the protocol agnostic `CConnman::m_unused_i2p_sessions`, its mutex and `MAX_UNUSED_I2P_SESSIONS_SIZE` to `SockMan`. Move `GetBindAddress()` from `net.cpp` to `sockman.cpp`.
Move `MaybeFlipIPv6toCJDNS()`, which is Bitcoin P2P specific from the callers of `CConnman::EventNewConnectionAccepted()` to inside that method. Move the IsSelectable check, the `TCP_NODELAY` option set and the generation of new connection id out of `CConnman::EventNewConnectionAccepted()` because those are protocol agnostic. Move those to a new method `SockMan::NewSockAccepted()` which is called instead of `CConnman::EventNewConnectionAccepted()`.
Move `CNode::m_sock` and `CNode::m_i2p_sam_session` to `SockMan::m_connected`. Also move all the code that handles sockets to `SockMan`. `CNode::CloseSocketDisconnect()` becomes `CConnman::MarkAsDisconnectAndCloseConnection()`. `CConnman::SocketSendData()` is renamed to `CConnman::SendMessagesAsBytes()` and its sockets-touching bits are moved to `SockMan::SendBytes()`. `CConnman::GenerateWaitSockets()` goes to `SockMan::GenerateWaitSockets()`. `CConnman::ThreadSocketHandler()` and `CConnman::SocketHandler()` are combined into `SockMan::ThreadSocketHandler()`. `CConnman::SocketHandlerConnected()` goes to `SockMan::SocketHandlerConnected()`. `CConnman::SocketHandlerListening()` goes to `SockMan::SocketHandlerListening()`.
`SockMan` members `AcceptConnection()` `NewSockAccepted()` `m_i2p_sam_session` `m_listen` are now used only by `SockMan`, thus make them private. `GetNewId()` is used by `SockMan` and in tests, so make it protected.
|
|
|
There are a lot of code moves in this PR. I put the following in my [diff]
colorMoved = dimmed-zebra
colorMovedWS = allow-indentation-changeit makes reviewing such mechanical moves easier. |
ryanofsky
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code review ACK 6091e94
I'm about halfway through reviewing this, and so far it seems like has a lot of nice code improvements in addition to conceptual benefits mentioned earlier #30988 (comment).
One thing I don't like about the API however, is the use of integer connection ids and introduction of hash maps to to tie connection ids, sockets, and application data together:
CConnman::m_listen_permissions- mapping listening addresses to permissionsCConnman::m_nodes- mapping connection ids to CNodesIOReadiness::ids_per_sock- mappingSockpointers to connection idsSockMan::m_connected``` - mapping connection ids toSock` pointers
I think it would be better to:
-
Keep the
ListeningSockstruct instead of getting rid of it, and give it astd::anymember so application data can be associated with it. The data could be passed toBindAndStartListeningandEventNewConnectionAccepted. -
Rename
struct ConnectionSocketstoclass Connectionand add astd::anymember to it that can hold theCNodereference. Delete theSockMan::Idtype, and useConnectionreferences everywhere ids are currently used.
I think this would provide a safer API and simplify code, allowing all the unordered maps to be dropped. Not very important though, since this PR is mostly moving code, and API improvements could be implemented in followups.
I plan to continue reviewing the PR.
| } | ||
|
|
||
| private: | ||
| NetPermissionFlags m_permissions; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In commit "net: separate the listening socket from the permissions" (1b2a87b)
It would seem less fragile to just replace the m_permissions field with a more generic field like std::any m_metadata or std::any m_user_data instead of requiring the application to maintain a separate unordered map and needing to deal with conditions when the map and vector could get out of sync (#30988 (comment), #30988 (comment)).
| #include <util/sock.h> | ||
|
|
||
| bool SockMan::BindListenPort(const CService& addrBind, bilingual_str& strError) | ||
| bool SockMan::BindAndStartListening(const CService& to, bilingual_str& err_msg) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In commit "style: modernize the style of SockMan::BindListenPort()" (6ba0aab)
Could be good to note in commit message what behaviors this is changing, so it clear they are intentional. After this commit:
- err_msg / strError output is no longer set if there are minor errors like failing to set SO_REUSEADDR
- "Bound to [address]" log message is dropped and moved to caller
- Other log messages are changed slightly
src/net.cpp
Outdated
| auto sock_accepted{AcceptConnection(*sock, addr_accepted)}; | ||
|
|
||
| if (sock_accepted) { | ||
| addr_accepted = MaybeFlipIPv6toCJDNS(addr_accepted); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In commit "net: split CConnman::AcceptConnection() off CConnman" (3e88116)
re: #30988 (comment)
Yes, good observation! Indeed the flip will do nothing on the invalid
0.0.0.0addr_accepted. [...]
IMO this information would be useful to mention in the commit message
| /// The listen succeeded and we are now listening for incoming I2P connections. | ||
| START_LISTENING, | ||
|
|
||
| /// The listen failed and now we are not listening (even if START_LISTENING was signaled before). | ||
| STOP_LISTENING, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In commit "net: move CConnman-specific parts away from ThreadI2PAcceptIncoming()" (07d731b)
Names and comments seem confusing and maybe inaccurate:
- The name STOP_LISTENING implies I2P was previously listening but now stopped, but the comment implies it might have never been listening.
- The name STOP_LISTENING seems like it could happen for reasons other than failing (like shutting down), so it's inconsistent with the comment which only mentions failing.
- Comments mention STOP could happen after START, but not that START could happen after STOP, even though the code seems to allow that.
- Comment implies these notifications are only sent on status changes, but code looks like these will be sent repeatedly even if nothing changes.
Would suggest rewriting and renaming to be clearer. Maybe:
// Indicates whether the I2P node is currently listening for new connections.
// A status update is sent on each accept attempt; it may be identical to the
// previous update, and the state may temporarily switch between listening and
// not listening.
enum class I2PStatus { LISTENING, NOT_LISTENING };|
|
||
| /** | ||
| * Constructor. | ||
| * @param[in] interrupt_net Singal this to tell SockMan to cease network activity. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In commit "net: move I2P-accept-incoming code from CConnman to SockMan" (3e91c8b)
s/Singal/Signal
| found = pnode; | ||
| break; | ||
| } | ||
| auto it{m_nodes.find(id)}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| * @retval >=0 The number of bytes actually sent. | ||
| * @retval <0 A permanent error has occurred. | ||
| */ | ||
| ssize_t SendBytes(Id id, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In commit "net: move sockets from CNode to SockMan" (a59b12a)
This commit is larger than the other commits. It would be nice if there's way to break it up since a few things are moving at the same time.
|
re: #30988 (comment)
After looking at #32061 more closely, I think I understand @theuni's concern about how the current SockMan interface does not work well for applications like the HTTP server. As I understand it, the problem in #32061 is that when an RPC method finishes and a reply is ready, the RPC thread has no clean way to wake up the SockMan thread and tell it there is data ready to send on this connection. The only thing The latest version of #32061 works around this partially by using the "optimistic send" pattern: IMO the SockMan interface could be improved to avoid this pitfall. One way would be to rename Alternately, if we go in the direction suggested in #30988 (review) and callbacks start passing #32061 probably shouldn’t need to care about any of this if it is rewritten to not use SockMan, but if we did want to generalize SockMan, we could think about adding a thread-safe |
|
🐙 This pull request conflicts with the target branch and needs rebase. |
Currently
CConnmanis a mixture of:AddrMan, maintaining the number of inbound and outbound connections, banning of peers, interacting withPeerManager.This PR splits the socket handling into a new class which makes the code more modular and reusable. Having more modular and reusable code is a good thing on its own, even if the code is not reused. Stratum V2 and libevent-less RPC/HTTP server could benefit from this, but it makes sense on its own, even without those projects.
The socket operations are driven by the new class
SockManwhich informs the higher level via provided methods when e.g. new data arrives on the socket or a new connection is accepted. For this,SockManprovides some non-virtual methods to start it rolling and then it calls pure virtual methods which are implemented by the higher level (e.g.CConnman) on certain events, for example "got this new data on this node's socket".The interface of
SockManis:Resolves: #30694
Review hint: this PR moves some code around, so reviewers may find this helpful:
git show --color-moved --color-moved-ws=allow-indentation-change.