Cloud Trace filters

You use trace filters to select a set of traces from Cloud Trace. Use a trace filter in one of the following ways:

As the filter parameter in a query using the list API method.
As the Request filter parameter when you create an analysis report..
As the Filter field in the Trace Explorer page. For information on creating and modifying these filters, see Display recent traces.

Note: The filter bar on the Trace Explorer page is menu-driven, and the options depend on your configuration. The menu-driven options typically include filtering by HTTP status and HTTP method. When you use App Engine, options for service and version are also available. These options aren't available using Cloud Trace filters.

Filters let you specify one or more conditions that must be satisfied for a trace to be displayed. The filter syntax lets you define when matches are exact, or when they are prefix tests. All matches are case-sensitive.

Filter overview

Trace filters consist of a sequence of comparisons called terms. Traces must match all the terms in the filter to be selected. For example, the following filter matches traces that have a latency of one second or more, and that contain a span named /modules.GetNumInstances:

+span:/modules.GetNumInstances latency:1s

All string comparisons are case-sensitive.

Boolean attributes

To specify values for Boolean attributes, use 0 for false, and 1 for true. For example,

Client:0

Special characters

To search for a value that contains a space or the colon (:) character, enclose the value in double-quote (") characters. For example,

    label:"Notice: This value contains spaces and a colon"

To search for the double-quote character or the backslash (\) character within a quoted string, escape the character with a backslash. For example,

    label:"Notice: This value contains spaces, a colon, a \"quote\", and a backslash (\\)"

Filter syntax

In the following table and elsewhere on this page, NAME refers to the name of a field while VALUE refers to the value of a key-value pair. The strings _PREFIX and _EXACT are visual indications of whether a test is a prefix-test or an exact match. For example, root:[NAME_PREFIX] means that the name of the root span must begin with [NAME_PREFIX].

The following table lists the syntax used filters in Cloud Trace:

Trace API Request Filter in the Analysis Reports page	Filter in the Trace Explorer page
`root:[NAME_PREFIX]`	`RootSpan:[NAME_PREFIX]`
`+root:[NAME_EXACT]`	`RootSpan:+[NAME_EXACT]`
`span:[NAME_PREFIX]`	`SpanName:[NAME_PREFIX]`
`+span:[NAME_EXACT]`	`SpanName:+[NAME_EXACT]`
`[NAME_PREFIX]`	`RootSpan:[NAME_PREFIX]`
`label:[LABEL_KEY]`	`HasLabel:[LABEL_KEY]`
`[LABEL_KEY]:[VALUE_PREFIX]`	`[LABEL_KEY]:[VALUE_PREFIX]`^*
`+[LABEL_KEY]:[VALUE_EXACT]`	`[LABEL_KEY]:+[VALUE]`^*
`^[LABEL_KEY]:[VALUE_PREFIX]`	`[LABEL_KEY]:^[VALUE_PREFIX]`^*
`+^[LABEL_KEY]:[VALUE_EXACT]`	`[LABEL_KEY]:^+[VALUE_EXACT]`^{*, †}
`method:[VALUE_PREFIX]`	`Method:[VALUE_PREFIX]`
`latency:10ms`	`MinLatency:10ms`
`url:[VALUE_PREFIX]`	`URL:[VALUE_PREFIX]`

^* Don't use the following Trace keywords as label keys: root, span, label, method, latency, and url. Traces with these label keys can't be queried.
^† In the Trace Explorer page, the annotation +^ is converted to ^+ when you press Enter.

The remainder of this page contains a description for filters when using Trace API and for the Request Filter on the Analysis Reports page. If you are using the Filter field in the Trace Explorer page, then you can use the mapping in the previous table to create an equivalent expression.

Request filter examples

The following describes the filter terms:

root:[NAME_PREFIX]

The trace's root-span name must begin with [NAME_PREFIX]. For example, the following filter matches a trace named "/_ah/background", but not one named "/_ahx/background":

root:/_ah/

+root:[NAME]

The trace's root-span name must be [NAME], exactly. For example:

+root:/_ah/background

[NAME_PREFIX]

This syntax is a shortcut for root:[NAME_PREFIX].

+[NAME]

This syntax is a shortcut for +root:[NAME].

span:[NAME_PREFIX]

The trace must have at least one span whose name begins with [NAME_PREFIX]. For example:

span:/modules.

+span:[NAME]

The trace must have at least one span whose name is [NAME], exactly. For example,

span:/modules.GetNumInstances

latency:[DURATION]

The trace must have an overall latency that is greater or equal to [DURATION]. The duration is expressed as an integer followed by a units specifier: s for seconds, ms for milliseconds, or ns for nanoseconds. If there is no unit specifier, then the units are milliseconds. For example, the following four durations are the same:

 12s 12000ms 12000000ns 12000

label:[LABEL_KEY]

The trace must contain the specified label key, exactly. The value of the label (if any) doesn't matter. For example,

label:/http/url

[LABEL_KEY]:[VALUE_PREFIX]

The trace must contain the specified label key, exactly, and the value of the label must start with [VALUE_PREFIX]. For example, the following term matches traces whose App Engine version begins with "2017".

g.co/gae/app/module_version:2017

+[LABEL_KEY]:[VALUE_EXACT]

The trace must contain the specified label key and value, exactly. For example,

+g.co/gae/app/module_version:201750925t173233.387410594824284458

method:[VALUE_PREFIX]

This syntax is a shortcut for the label test, /http/method:[VALUE_PREFIX].

+method:[VALUE_EXACT]

This syntax is a shortcut for the label test, +/http/method:[VALUE_EXACT].

url:[VALUE_PREFIX]

This syntax is a shortcut for label test, /http/url:[VALUE_PREFIX].

+url:[VALUE_EXACT]

This syntax is a shortcut for label test, +/http/url:[VALUE_EXACT].

Root-span search examples

To restrict the search term to only the root span, add a (^) before a search term and after the optional (+). For the Filter field in the Trace Explorer page, if you add the annotation +^, it is converted to ^+ when you press Enter.

The following show some root-span-only search terms and their equivalences:

^label:[LABEL_KEY]: This is a root-span search for label:[LABEL_KEY].
^[LABEL_KEY]:[VALUE_PREFIX]: This is a root-span search for [LABEL_KEY]:[VALUE_PREFIX].
+^[LABEL_KEY]:[VALUE]: This is a root-span search for +[LABEL_KEY]:[VALUE].
^method:[VALUE_PREFIX]: This is a root-span search for method:[VALUE_PREFIX].
+^method:[VALUE]: This is a root-span search for +method:[VALUE].
^url:[VALUE_PREFIX]: This is a root-span search for url:[VALUE_PREFIX].
+^url:[VALUE]: This is a root-span search for +url:[VALUE].
^span:[NAME_PREFIX]: This is equivalent to root:[NAME_PREFIX].
+^span:[NAME]: This is equivalent to +root:[NAME].

By creating a query with mixed terms, the search can be further customized. For example,

    +^url:/main /images method:200

matches traces when the root span label /http/url is exactly "/main", the root span's name is prefixed by /images, and any span has the label /http/method with a value of 200.

Troubleshooting

This section contains troubleshooting content.

Slow performance

If your trace filter is complicated, it takes longer to run and could time out. To improve performance, simplify the filter.

No results

If your filter does not return any traces, check the following:

Be sure there are no spaces in the filter except the ones that separate the comparisons.
Be sure the letter case and spelling of all the words in the filter is correct. For example, if you misspell a keyword such as method:GET, the filter is interpreted as label:method:GET, which does not match any traces.
To test the filter terms one at a time, use the Trace List filter or the API Explorer. If one of the terms returns no results, it might be the culprit.