Skip to content

Memory leak on Node 19/20 with includeLocalVariables enabled #14162

Open

Description

Is there an existing issue for this?

How do you use Sentry?

Sentry Saas (sentry.io)

Which SDK are you using?

@sentry/node

SDK Version

8.36.0

Framework Version

Node 19.9.0, also tested on Node 20.18.0

Link to Sentry event

N/A

Reproduction Example/SDK Setup

import * as Sentry from "@sentry/node"

import api from "@opentelemetry/api"



function trimData(data) {
  const size = JSON.stringify(data || "").length

  // Arbitrarily cut off massive payloads. Sentry silently drops events that are too large,
  // so trim them.
  // See https://github.com/getsentry/sentry-javascript/issues/3036#issuecomment-1066491190
  if (size > 200000) {
    return data?.slice(0, 200000)
  }

  return data
}

function checkIfSentryIgnore(url) {
  if (!url) return false

  for (const route of SENTRY_IGNORED_ROUTES) {
    if (typeof route === "string" && url.endsWith(route)) {
      return true
    } else if (route instanceof RegExp && route.test(url)) {
      return true
    }
  }
}

/**
 * Need to ignore long-running websockets.
 *
 * See: https://github.com/getsentry/sentry-javascript/issues/13412#issuecomment-2340434248
 */
const IGNORE_WEBSOCKETS = ["infura", "ws", "alchemy", "ankr", "zora"]

Sentry.init({
  dsn: "snip",

  // Disable entirely on non-production environments.
  enabled: true,
  debug: !!process.env.SENTRY_DEBUGGING,
  registerEsmLoaderHooks: { exclude: [/openai/] },
  includeLocalVariables: true,
  integrations: [
    Sentry.onUncaughtExceptionIntegration({
      exitEvenIfOtherHandlersAreRegistered: false,
    }),
    Sentry.expressIntegration(),
    Sentry.httpIntegration({
      ignoreOutgoingRequests(url, request) {
        console.log("SENTRY: should we ignore outgoing request?", url)
        for (const route of IGNORE_WEBSOCKETS) {
          if (url.includes(route)) {
            console.log("SENTRY: Ignoring outgoing request", url)
            return true
          }
        }

        console.log("SENTRY: Not ignoring outgoing request", url)
        return false
      },
    }),
  ],

  release: process.env.COMMIT,
  environment: process.env.FLY_APP_NAME || "development",

  beforeSendTransaction(event) {
    console.log("SENTRY: beforeSendTransaction", event)

    if (event.transaction === "redis-connect") {
      // Don't send the event to Sentry
      return null
    }

    console.log("SENTRY: Sending transaction", event)

    return event
  },
  beforeSend: (event) => {
    console.log("SENTRY: beforeSend", event)
    try {
      if (event.request?.data) {
        event.request.data = trimData(event.request.data)
      }
      if (event.message) {
        event.message = trimData(event.message)
      }

      return event
    } catch (e) {
      console.error("Error in Sentry beforeSend", e, event)
      return event
    }
  },
})

Steps to Reproduce

Run Node 19 or Node 20 with Sentry enabled in my specific setup. The memory gradually increases until an OOM occurs.

This graph shows when Node was updated from 18 to 19.9.0. No other code changes occurred.
Image

Expected Result

Memory does not gradually increase until an OOM occurs.

This graph shows when I disabled includeLocalVariables in the above Sentry.init: memory stops increasing.
Image

Actual Result

Memory increases gradually until it OOMs, if includeLocalVariables is enabled.

If Sentry is disabled entirely, or if the includeLocalVariables flag is disabled, then memory looks as expected.

I originally suspected this was the issue, since we also use websockets and long-running requests, but after ignoring the requests and confirming via the console.logs that none of the long-running requests got spans created (with the issue still persisting), I tried disabling local variables and that fixed it.

These are (truncated) logs with trace GC enabled. It's not particularly showing much other than memory growth. This shows with local variables enabled, and this is with it disabled.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

Labels

Package: nodeIssues related to the Sentry Node SDK

Type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions