-
Notifications
You must be signed in to change notification settings - Fork 17.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
runtime: add AddCleanup #67535
Comments
Another possibility:
When cleaning up, we pass a (shallow) copy of This gets rid of the need to store On the downside, this may get weird if you're putting a cleanup on a type defined in another package, as you might not be able to do much with the argument. |
@mknyszek I find it hard from reading the proposed signatures and doc comments to understand what Another question: would |
What's the reasoning here? This means that you can not assume that If this was done the other way around - cleanups run after finalizers and only if Or is there something (perhaps in the implementation of the runtime) that I'm unaware of? |
Unfortunately, I think that requires treating the contents of
The idea behind this API is to decouple cleanup from the object being freed, so
I'm taking a lot of liberties in this snippet, but that would be the general idea. I can update the proposal later. |
That's a good point.
What about this case?
During the lifetime of a How do I write a cleanup for that? I think I would need to Stop/AddCleanup each time I updated |
@Merovius I think your point about interacting with existing objects using finalizers is interesting. AFAIK there's nothing in the implementation preventing the semantics you propose. I'll have to give this some more thought. |
Ah, that's interesting. Yeah, you would need to I'm inclined to say that requiring the indirection isn't that bad. It's going to result in an additional allocation for each |
The indirect scheme is basically the same thing you would do with finalizers if, e.g., the T's were in a cycle. You would allocate a child object, point T to it, put Having the option to Stop/AddCleanup each time instead of using the indirection technique might be useful. I'm not sure I'm arguing for or against anything at this point. Just trying to understand. |
What about deferCleanup as a name. Defer is already very understandable by Go developers (albeit in a different context). Add doesn't really say anything about when it happens. |
Ruby does something similar for finalizers. Basically the finalizer is run after the object is deallocated. They are also guaranteed to run if the object is deallocated. The Ruby finalizer is not allowed to reference the object that it finalizes. In stead you have to reference to any members through a closure. In my experience this works better than Go finalizers. I would recommend Go adopts the Ruby way of implementing finalizers. |
AddCleanup
and deprecate SetFinalizer
Here is a nice (imho) use of AddCleanup where SetFinalizer would not be okay: #67552 (comment). |
This proposal has been added to the active column of the proposals project |
The SetFinalizer documentation specifies that the function runs in a new go routine, I think this API documentation should also specify that (as long as that is indeed the case). |
Indeed. Read sec 3.5 of Destructors, Finalizers, and Synchronization, in which Hans Boehm argues that finalizers must be asychronous, for the rationale. |
@perj This is maybe a little pedantic, but it's not quite true that it runs in a new goroutine. :) From the
I agree that we should copy this part of the docs to We do have the option here of changing the behavior of cleanups to always start on a new goroutine. It's more user-friendly (in that blocking in a cleanup function won't delay other cleanups), but may result in greater resource costs. I do think a valid question is whether we want to retain the existing sequential behavior. Perhaps one middle-ground is to relax the constraints and mostly reuse the same goroutine, but if a cleanup goroutine blocks, then we start a new one. My only concern with this is that data races between cleanup functions may be much harder to detect if they're not fully asynchronous by default. |
@mknyszek If a new goroutine is desired for a finalizer it can always |
Just to summarize, here are the three choices a described above as well as their pros/cons. There may be more options. Guaranteed sequential execution of cleanups (What
Cleanups always run in a new goroutine
Cleanups may run in a new goroutine
|
SetFinalizer has many footguns but I don't think I've seen many people complain about the fact that one can block others. It seems okay to leave just a single goroutine. I did a tiny bit of wordsmithing in the comment. Reading the SetFinalizer comment, there is a lot of good advice and caveats that seem to carry over (zero-size allocations, small pointer-free allocations, globals, not using them for flushing buffers, how to use KeepAlive properly, memory model interactions). Let's not hash them out in the proposal process, but the real implementation's doc comment should probably carry them forward. |
Have all remaining concerns about this proposal been addressed? The details are in the top comment under the "Proposal" heading. |
I think the only remaining question is whether cleanups should run before or after finalizers when both are present. @Merovius made a case for running cleanups after any attached finalizer completes, so that they can rely on the object they're attached to being truly dead. If there are no finalizers present, only cleanups and/or weak pointers, then the cleanups still get to run immediately and everything is great. This is different from the semantics proposed for weak pointers, which are to go nil the first time an object becomes unreachable since the weak pointer was created. Meanwhile, cleanups would track object resurrection, but not resurrect objects themselves. So far, I've been unable to come up with a situation where running cleanups after finalizers would be problematic, but I think there is a case to be made that running cleanups before finalizers would be problematic. But, I suspect cleanups don't have the same problem as weak pointers with respect to tracking resurrection, because there's no way for cleanups to reference the object they're attached to. So, I guess I'm inclined to just change that one part of the proposal. If anyone has additional thoughts, I'd appreciate them. I'll update the proposal at the top of this issue. |
I don't have an opinion, but I want to at least raise another possibility: calling Both of these functions are touchy in practice, and in the common case of releasing some external resource require careful use of |
ISTM that such a restriction would make it dangerous to use AddCleanup on objects not "owned" by the caller, because they cannot control whether SetFinalizer has been called on the objects or not. |
Is this planned for Go 1.2 |
@mknyszek |
Change https://go.dev/cl/627695 mentions this issue: |
Change https://go.dev/cl/627975 mentions this issue: |
No, the call to In short, (We could extend this API to support slices, and we could tell people to do some weird unsafe conversion to a |
This change introduces AddCleanup to the runtime package. AddCleanup attaches a cleanup function to an pointer to an object. The Stop method on Cleanups will be implemented in a followup CL. AddCleanup is intended to be an incremental improvement over SetFinalizer and will result in SetFinalizer being deprecated. For #67535 Change-Id: I99645152e3fdcee85fcf42a4f312c6917e8aecb1 Reviewed-on: https://go-review.googlesource.com/c/go/+/627695 LUCI-TryBot-Result: Go LUCI <[email protected]> Reviewed-by: Michael Knyszek <[email protected]>
This change adds the implementation for AddCleanup.Stop. It allows the caller to cancel the call to execute the cleanup. Cleanup will not be stopped if the cleanup has already been queued for execution. For #67535 Change-Id: I494b77d344e54d772c41489d172286773c3814e5 Reviewed-on: https://go-review.googlesource.com/c/go/+/627975 LUCI-TryBot-Result: Go LUCI <[email protected]> Reviewed-by: Michael Knyszek <[email protected]> Auto-Submit: Carlos Amedee <[email protected]>
Change https://go.dev/cl/629215 mentions this issue: |
This proposal includes the deprecation of |
Background
Go provides one function for object finalization in the form of
runtime.SetFinalizer
. Finalizers are notoriously hard to use, and the documentation ofruntime.SetFinalizer
describes all the caveats with a lot of detail. For instance:SetFinalizer
must always refer to the first word of an allocation. This means programmers must be aware of what an 'allocation' is whereas that distinction isn't generally exposed in the language.The last two of these caveats boil down to the fact that
runtime.SetFinalizer
allows object resurrection.Proposal
I propose adding the following API to the
runtime
package as a replacement forSetFinalizer
. I also propose officially deprecatingruntime.SetFinalizer
.AddCleanup
resolves many of the problems withSetFinalizer
.It forbids objects from being resurrected, resulting in prompt cleanup, as well as allowing cycles of objects to be cleaned up. Its definition also allows attaching cleanup functions to objects the caller does not own, and possibly attaching multiple cleanup functions to a single object.
However, it is still fundamentally a finalization mechanism, so to avoid restricting the GC implementation, it does not guarantee that the cleanup function will ever run.
Similar to finalizers' restriction on the object not being reachable from the finalizer function,
ptr
must not be reachable from the value passed to the cleanup function, or from the cleanup function. Usually this results in a memory leak, but the common case of accidentally passingptr
ass
out of convenience can be easily caught.In terms of interactions with finalizers, the cleanup function will run after the value pointed to by
ptr
becomes unreachable and its finalizer has run (and that finalizer does not make the object reachable again). That is, if an object has both a cleanup function and a finalizer, the cleanup function is guaranteed to run after the finalizer. In other words, the cleanup function tracks object resurrection.Design discussion
Avoiding allocations in the implementation of
AddCleanup
AddCleanup
needs somewhere to storecleanupValue
untilcleanup
is ready to be called. Naively, it could just put that value in anany
variable somewhere, but this would result in an unnecessary additional allocation.In the actual implementation, a cleanup will be represented as a runtime "special," an off-heap manually-managed linked-list node data structure whose individual fields are sometimes explicitly inspected by the GC as roots, depending on the "special" type (for example, a finalizer special treats the finalizer function as a root).
Since each "special" is already specially (ha) treated by the GC, we can play some interesting tricks. For example, we could type-specialize specials and store
cleanupValue
directly in the "special." As long as we retain the type information forcleanupValue
, we can get the GC to scan it directly in the special. But this is quite complex.To begin with, I propose just specializing for word-sized cleanup values. If the cleanup value fits in a pointer-word, we store that directly in the special, and otherwise fall back on the equivalent of an
any
. This would cover many use-cases. For example, cleanup values that are already heap-allocated pointers wouldn't require an additional allocation. Also, simple cases like passing off a file descriptor to a cleanup function would not create an allocation.Why
func(S)
and notfunc()
?The choice to require an explicit parameter to the cleanup function is to reduce the risk of the cleanup function accidentally closing over
ptr
. It also makes it easier for a caller to avoid allocating a closure for each cleanup.Why
func(S)
and notchan S
?Channels are an attractive alternative because they allow users to build their own finalization queues. The downside however is that each channel owner needs its own goroutine for this to be composable, or some third party package needs to exist to accumulate all these channels and select over them (likely with reflection). It's much simpler if that package is just the runtime: there's already a system goroutine to handle the finalization queue. While this does mean that the handling of finalization is confined to an implementation detail, that's rarely an issue in practice and having the runtime handle it is more resource-efficient overall.
Why return
Cleanup
instead of*Cleanup
?While
Cleanup
is a handle and it's nice to represent that handle's unique identity with an explicit pointer, it also forces an allocation ofCleanup
's contents in many cases. By returning a value, we can avoid an additional allocation.Why not have the type arguments (
T
and/orS
) onCleanup
too?It's not necessary for the implementation of
Cleanup
for the type arguments to be available, since the internal representation will not even contain a reference toptr
,cleanup
, orcleanupValue
directly. It does close the door to obtaining these values fromCleanup
in a type-safe way, but that's OK: the caller ofAddCleanup
can already package those up together if it wants to.The text was updated successfully, but these errors were encountered: