Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a globalbalance API call #1737

Merged
merged 1 commit into from
Jul 8, 2021
Merged

Add a globalbalance API call #1737

merged 1 commit into from
Jul 8, 2021

Conversation

pm47
Copy link
Member

@pm47 pm47 commented Mar 23, 2021

For now it just gives detailed info on the current pending mutual closes.

@codecov-commenter
Copy link

codecov-commenter commented Jun 1, 2021

Codecov Report

Merging #1737 (4b78392) into master (85ed433) will decrease coverage by 0.70%.
The diff coverage is 47.66%.

@@            Coverage Diff             @@
##           master    #1737      +/-   ##
==========================================
- Coverage   88.94%   88.24%   -0.71%     
==========================================
  Files         150      154       +4     
  Lines       11239    11417     +178     
  Branches      447      426      -21     
==========================================
+ Hits         9997    10075      +78     
- Misses       1242     1342     +100     
Impacted Files Coverage Δ
...ain/scala/fr/acinq/eclair/balance/Monitoring.scala 0.00% <0.00%> (ø)
.../main/scala/fr/acinq/eclair/channel/Register.scala 81.48% <ø> (ø)
...r-core/src/main/scala/fr/acinq/eclair/Eclair.scala 47.09% <6.25%> (-5.05%) ⬇️
...n/scala/fr/acinq/eclair/balance/BalanceActor.scala 16.66% <16.66%> (ø)
...n/scala/fr/acinq/eclair/balance/CheckBalance.scala 57.83% <57.83%> (ø)
...re/src/main/scala/fr/acinq/eclair/NodeParams.scala 92.89% <100.00%> (+0.03%) ⬆️
...ir-core/src/main/scala/fr/acinq/eclair/Setup.scala 80.45% <100.00%> (+0.57%) ⬆️
...ala/fr/acinq/eclair/balance/ChannelsListener.scala 100.00% <100.00%> (ø)
.../acinq/eclair/blockchain/bitcoind/ZmqWatcher.scala 95.86% <100.00%> (-1.01%) ⬇️
...lockchain/bitcoind/rpc/ExtendedBitcoinClient.scala 94.11% <100.00%> (+0.17%) ⬆️
... and 10 more

@pm47 pm47 force-pushed the balance-api branch 2 times, most recently from 0452c6e to 6a756c4 Compare June 3, 2021 08:33
@pm47 pm47 marked this pull request as ready for review June 29, 2021 14:13
@pm47 pm47 requested a review from t-bast June 29, 2021 14:13

import scala.concurrent.Promise

object ChannelsListener {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The goal of this actor is to maintain a local cache of all channel data, that can be used by the API, instead of querying thousands of channel actors. Surprisingly, this is turns out to not be noticeably faster than the ask+Future.sequence pattern.

Maybe we should remove this? Or we could keep it and use that for all remaining API calls, because it saves some burden of the channel actors? I'm not sure.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's useful to have it, when we'll have more load on the channel actors it should be noticeably useful, don't you think?

@pm47
Copy link
Member Author

pm47 commented Jun 29, 2021

Squashed and rebased.

Copy link
Member

@t-bast t-bast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First pass, I haven't looked in details in the actual balance computation, but I'm proposing a few architectural changes and found some nits.

eclair-core/src/main/scala/fr/acinq/eclair/Eclair.scala Outdated Show resolved Hide resolved
@@ -53,15 +57,31 @@ case class AuditResponse(sent: Seq[PaymentSent], received: Seq[PaymentReceived],

case class TimestampQueryFilters(from: Long, to: Long)

case class MutualCloseStatus(unpublished: Satoshi, unconfirmed: Satoshi, confirmed: Satoshi)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: MutualCloseBalance would be more consistent IMO

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, this is just additional information about mutual close transactions, useful to track evictions. The mutual close balance is tracked as part of onchain balance, see:

  /**
   * Mutual close transactions are always immediately published, and will be taken into account by bitcoin core, so we
   * don't count them in the balance.
   */
  case class ClosingBalance

But now that I think of it, we should probably treat mutual closes like other transactions, in the offchain balance, and prune it afterwards. The goal of pruning was initially to remove duplicates for transactions that were published (and possibly confirmed but less than mindepth), but it should also nicely handle evicted transactions.

Copy link
Member Author

@pm47 pm47 Jul 1, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I remember now why the mutual close transactions were handled like this. The problem is that there can be several competing mutual closes, and in that case the simple pruning logic that we have to make sure we count the amount exactly once doesn't work. This would lead us to possibly count twice the amount in the competing-mutual-closes scenario.

On the other hand, what we had before meant that we didn't count the amount in the case where the mutual close was evicted. Not perfect either, and this is why we had the additional MutualCloseStatus item.

In general I would say that it is better to undershoot rather than overshoot the balance computation, but here the trade-off makes sense for consistency, and also it is easy to tell when we are potentially counting a mutual close twice: the mutualCloseBalance should always be zero after pruning.

So I went ahead and did it with 4b78392, with a detailed comment on what is going on.

eclair-core/src/main/scala/fr/acinq/eclair/Eclair.scala Outdated Show resolved Hide resolved
@pm47 pm47 requested a review from t-bast July 1, 2021 15:27
@@ -2370,6 +2374,24 @@ class NormalStateSpec extends TestKitBaseClass with FixtureAnyFunSuiteLike with
// at best we have a little less than 450 000 + 250 000 + 100 000 + 50 000 = 850 000 (because fees)
assert(amountClaimed === 814880.sat)

val commitments = alice.stateData.asInstanceOf[DATA_CLOSING].commitments
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can leave it like that for now, but I'm not a big fan of adding new unrelated checks inside existing tests (especially when these tests are already quite long and complex).

It's a bit more code, but I would not put any balance test inside this file, I would instead write these tests in CheckBalanceSpec (where you would setup channels and HTLCs similarly to what's done here, but with the flexibility of changing the scenarios tested without impacting the rest of the test).

t-bast
t-bast previously approved these changes Jul 5, 2021
Copy link
Member

@t-bast t-bast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I haven't looked in details at the actual balance computation details, real-world experience is the best judge for whether it works or not ;)

And we can easily implement several balance computation algorithms and run them in parallel, and see which one mirrors more closely reality.

It returns an overall balance, separating onchain, offchain, and
removing duplicates (e.g. mutual closes that haven't reached min depth
still have an associated channel, but they already appear in the
on-chain balance). We also take into account known preimages, even if
the htlc hasn't been formally resolved.

Metrics have also been added.

fixup: typo

Co-authored-by: Bastien Teinturier <[email protected]>

better serializers

naming consistency

treat mutual close transactions like other closes

move utxo metrics to balance actor

Also refactor to make it consistent with the existing actor.

NB: there is still duplication in the metrics, will handle that
separately.

rename UnlockOutpointUtxo -> UnlockOutpoint

refactor interaction between api and balance actor

fixed half-finished comment

improve the event stream synchronization trick

remove left-over logs

Co-authored-by: Bastien Teinturier <[email protected]>

make GetGlobalBalance return a Try, not a Future

make merge easier in feature branches
@pm47
Copy link
Member Author

pm47 commented Jul 7, 2021

Rebased and squashed.

@pm47 pm47 merged commit bd57d41 into master Jul 8, 2021
@pm47 pm47 deleted the balance-api branch July 8, 2021 11:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants