Monitor workflows

This page provides information to help you monitor workflow deployments and executions.

Access workflow deployment and deletion logs

You can access error logs related to a workflow's deployment and deletion in the Google Cloud console.

1. In the Google Cloud console, go to the Workflows page.

   Go to Workflows

2. Click the workflow's name to display its Workflow details page.

3. Click the Logs tab.

4. To filter the logs by severity, in the Default list, select the type of log to display.

Access workflow execution results

You can access workflow execution results in the Google Cloud console or by using the gcloud CLI.

Execution error maps

When a workflow throws an error during execution that isn't caught in a try/except block, the execution fails, and an error map (a JSON dictionary) describing the error is returned.

Errors thrown during workflow execution contain tags to help you identify what caused the error. For example, the error returned from a connector can have two keys (tags and message) similar to the following:

{'tags': ['SystemError'], 'message': 'an error has occurred'}

There can be more than one tag. To check for a specific tag, you can use an expression. For example:

${'SystemError' in e.tags}

Access error data returned as a string

Some connectors and HTTP APIs will serialize errors as strings before returning the errors. You can use standard library functions to restore a payload to the original error. For example, to convert an error string to a map, you can use the json.decode and text.encode functions:

json.decode(text.encode(ERROR_FROM_API))

Error tags

The following table describes the meaning of different error tags.

You can also raise custom errors using the raise syntax.

Check the status of executions

There are several commands to help you check the status of a workflow execution.

• To retrieve a list of a workflow's execution attempts and their IDs, enter the following command:

  gcloud workflows executions list WORKFLOW_NAME

  Replace WORKFLOW_NAME with the name of the workflow.

  The command returns a NAME value that is similar to the following:

  projects/PROJECT_NUMBER/locations/REGION/workflows/WORKFLOW_NAME/executions/EXECUTION_ID

  Copy the execution ID to use in the next command.

• To check the status of an execution attempt and wait for the attempt to finish, enter the following command:

  gcloud workflows executions wait EXECUTION_ID

  Replace EXECUTION_ID with the execution attempt's ID.

  The command waits for the execution attempt to finish and then returns the results.

• To wait until the last execution is complete and then return the result of the completed execution, enter the following command:

  gcloud workflows executions wait-last

  If you made a previous execution attempt in the same gcloud session, the command waits for the prior execution attempt to finish and then returns the results of the completed execution. If no previous attempt exists, gcloud returns the following error:

  ERROR: (gcloud.workflows.executions.wait-last) [NOT FOUND] There are no cached executions available.
  

• To get the status of the last execution, enter the following command:

  gcloud workflows executions describe-last

  If you made a previous execution attempt in the same gcloud session, the command returns the results of the last execution even if it is running. If no previous attempt exists, gcloud returns the following error:

  ERROR: (gcloud.beta.workflows.executions.describe-last) [NOT FOUND] There are no cached executions available.
  

Filter executions

You can apply filters to the list of workflow executions returned by the workflows.executions.list method.

You can filter on the following fields:

• createTime
• disableOverflowBuffering
• duration
• endTime
• executionId
• label
• startTime
• state
• stepName
• workflowRevisionId

For example, to filter on a label (labels."fruit":"apple"), you can make an API request similar to the following:

GET https://workflowexecutions.googleapis.com/v1/projects/MY_PROJECT/locations/MY_LOCATION/workflows/MY_WORKFLOW/executions?view=full&filter=labels.%22fruit%22%3A%22apple%22"

Where:

• view=full specifies a view defining which fields should be filled in the returned executions; in this case, all the data
• labels.%22fruit%22%3A%22apple%22is the URL-encoded filter syntax

For more information, see AIP-160 Filtering.

Send logs to Cloud Logging

Workflows automatically generates execution logs for workflow executions in Cloud Logging.

You can also enable call logging. Or, you can create custom logs that use the sys.log function in your source. Call logging and custom logs let you control when logs are sent to Logging during a workflow execution and can be particularly helpful when debugging your workflow.

For details, including the engine_call and executions_system logging proto files, see this GitHub repository.

Execution logs

Each workflow execution automatically triggers at least two execution logs: one at the start of an execution and one at the end.

For more information about the Workflows platform logs that are available in Logging, see Google Cloud platform logs.

Call logging

You can set a flag so that each call step during the execution of your workflow is logged, and step names, function names, function arguments, and call responses are returned. Or, you can log any exceptions that are caught, or that stop a call.

Only explicit call steps are logged; for example, calls to subworkflows or library functions. Calls from within expressions, or within standard library functions (for example, http.post in sys.log), and inside connectors are not logged.

HTTP Authorization request headers are redacted from the logs for HTTP calls.

When applying call logging to a workflow definition or to the execution of a workflow, you can specify the level of logging required. The execution log level takes precedence over any workflow log level, unless the execution log level is not specified (the default); in that case, the workflow log level applies.

Note that the log entry size limit set by Cloud Logging also applies to call logging.

Custom logs

To create a log entry in Logging during a workflow execution, define a step in the workflow that makes a call to the standard library sys.log function:

  - step1:
      assign:
          - varA: "Hello"
          - varB: "World"
  - logStep:
      call: sys.log
      args:
          text: TEXT
          severity: SEVERITY 
  - step2:
      return: ${varA + " " + varB}
    

    [
      {
        "step1": {
          "assign": [
            {
              "varA": "Hello"
            },
            {
              "varB": "World"
            }
          ]
        }
      },
      {
        "logStep": {
          "call": "sys.log",
          "args": {
            "text": "TEXT",
            "severity": "SEVERITY"
          }
        }
      },
      {
        "step2": {
          "return": "${varA + " " + varB}"
        }
      }
    ]
      

When creating a log entry, define the following:

• TEXT: required. The text to be logged. If you need to log the values of a map, use ${json.encode_to_string(myMap)}.
• SEVERITY: optional. The severity level of the log entry. For example, INFO, WARNING, or CRITICAL.

For more information, see the sys.log function reference.

Required permissions

To apply call logging or send custom logs to Logging, a workflow must be associated with a service account that includes the logging.logEntries.create permission (for example, the roles/logging.logWriter role). If you need to change the service account updated with your workflow, see Update a workflow. To learn more about creating service accounts and assigning roles, see Manage access to projects, folders, and organizations.

View workflow logs

You can view logs in Workflows or in Logging. To view the logs for a single workflow, use the Logs tab in Workflows. To get an aggregate view of the logs for all of your workflows, use the Logs Explorer page in Logging.

View logs in Workflows

To view a workflow's logs in Workflows, do the following:

1. In the Google Cloud console, go to the Workflows page:

   Go to Workflows

2. To access a workflow's logs, click the workflow's name to go to its Details page.

3. To view the logs, click Logs.

4. To filter the logs by severity, in the Default list, select the type of log to display. By default, logs of all severity levels are shown.

The Logs tab on a workflow's Details page displays the following types of logs:

• Logs sent to Logging

• Audit logs of any operations performed on the workflow, such as updates to the workflow's definition

View logs in Logging

To view logs in Logging, do the following:

1. In the Google Cloud console, go to the Logs Explorer page:
   

   Go to Logs Explorer

2. In the Query builder, click Resource and enter workflow. Select Cloud Workflow from the list and click Add.

   

3. Click Run query.

To learn more about viewing logs in Logging, see Use the Logs Explorer.

What's next

• Debugging overview
• Troubleshoot issues
• Known issues for Workflows