-
Notifications
You must be signed in to change notification settings - Fork 310
Closed
Description
As #391 noticed - docker_auth is leaking memory by leaking goroutines.
Looking at the heap profile runtime.malg is using a lot of memory, this is because of a goroutine leak:
$ go tool pprof 2024-09-07T16:13:49+1000_goroutine.out
File: auth_server
Build ID: f0023e9b8e89438ff941b55f3266cd69dfff1f88
Type: goroutine
Time: Sep 7, 2024 at 4:13pm (AEST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top5
Showing nodes accounting for 10539, 100% of 10542 total
Dropped 45 nodes (cum <= 52)
Showing top 5 nodes out of 9
flat flat% sum% cum cum%
10539 100% 100% 10539 100% runtime.gopark
0 0% 100% 4889 46.38% github.com/schwarmco/go-cartesian-product.Iter.func1
0 0% 100% 5638 53.48% github.com/schwarmco/go-cartesian-product.iterate
0 0% 100% 5638 53.48% runtime.chansend
0 0% 100% 5638 53.48% runtime.chansend1
(pprof)I also created an issue against schwarmco/go-cartesian-product#6 with the potential solution, but posting it here for visibility as well:
diff --git a/auth_server/authz/acl.go b/auth_server/authz/acl.go
index 2f2e824..b0aa21c 100644
--- a/auth_server/authz/acl.go
+++ b/auth_server/authz/acl.go
@@ -1,6 +1,7 @@
package authz
import (
+ "context"
"encoding/json"
"fmt"
"net"
@@ -11,7 +12,6 @@ import (
"strings"
"github.com/cesanta/glog"
- "github.com/schwarmco/go-cartesian-product"
"github.com/cesanta/docker_auth/auth_server/api"
)
@@ -180,14 +180,17 @@ func matchStringWithLabelPermutations(pp *string, s string, vars []string, label
}
}
if len(labelSets) > 0 {
- for permuation := range cartesian.Iter(labelSets...) {
+ ctx, cancel := context.WithCancel(context.Background())
+ defer cancel()
+
+ for permuation := range IterWithContext(ctx, labelSets...) {
var labelVars []string
for _, val := range permuation {
labelVars = append(labelVars, val.([]string)...)
}
matched = matchString(pp, s, append(vars, labelVars...))
if matched {
- break
+ return matched
}
}
}
@@ -195,6 +198,45 @@ func matchStringWithLabelPermutations(pp *string, s string, vars []string, label
return matched
}
+func IterWithContext(ctx context.Context, params ...[]interface{}) <-chan []interface{} {
+ c := make(chan []interface{})
+
+ if len(params) == 0 {
+ close(c)
+ return c
+ }
+
+ go func() {
+ defer close(c) // Ensure the channel is closed when the goroutine exits
+
+ iterate(ctx, c, params[0], []interface{}{}, params[1:]...)
+ }()
+
+ return c
+}
+
+func iterate(ctx context.Context, channel chan []interface{}, topLevel, result []interface{}, needUnpacking ...[]interface{}) {
+ if len(needUnpacking) == 0 {
+ for _, p := range topLevel {
+ select {
+ case <-ctx.Done():
+ return // Exit if the context is canceled
+ case channel <- append(append([]interface{}{}, result...), p):
+ }
+ }
+ return
+ }
+
+ for _, p := range topLevel {
+ select {
+ case <-ctx.Done():
+ return // Exit if the context is canceled
+ default:
+ iterate(ctx, channel, needUnpacking[0], append(result, p), needUnpacking[1:]...)
+ }
+ }
+}
+
func matchIP(ipp *string, ip net.IP) bool {
if ipp == nil {
return trueI think the function is small and simple enough you can copy it into your code and not rely on a lib. Let me know if you want me to raise a PR with the above changes.
Thanks!
Metadata
Metadata
Assignees
Labels
No labels