Configure observability scopes

This document describes how you can configure one Google Cloud project to monitor or display telemetry data from multiple Google Cloud projects. If you only want to monitor or view data that is stored in one Google Cloud project, then you don't need to perform any configuration, as the visualization and analysis tools are configured to use the data stored in the Google Cloud project selected by the project picker. However, if the telemetry data that you want to view or analyze is from multiple projects, then to have an aggregated view of that data, you must perform some configuration activities.

About observability scopes

The Google Cloud Observability analysis and visualization tools rely on data-type specific scopes to determine what data to display or analyze. A scope defines the resources that are searched for a particular type of data. With the exception of metric data, your Identity and Access Management (IAM) roles on the searched projects and log views determine what data is displayed.

The remainder of this section describes the scopes you can view and configure.

Observability scope

This scope controls how explorer and dashboard pages search for the data to display. Each Google Cloud project contains a single observability scope, which lists the default log scope, the metrics scope, and the default trace scope.

We recommend configuring the components of the observability scope in the following scenarios:

  • You register applications with App Hub. These might be applications you register yourself, or those you deployed by using the Application Design Center.
  • When you want a unified view of the telemetry data that is stored in different Google Cloud projects.

You don't have to configure the components of the observability scope for a project. If you rely on system defaults, then the following occurs:

  • The Logs Explorer page displays log data that originates in the project.
  • The Metrics Explorer and Trace Explorer pages display the metric and trace data stored in the project.
  • Alerting policies monitor metric data stored in the project.

Log scopes

Configure the default log scope of a project so that when you open the Logs Explorer page, the data that you usually want to view is displayed. A log scope can list projects, folders, organizations, and log views. For example, you might set the default log scope to list a log view, which when queried, returns the log data for an App Hub application.

You can create multiple log scopes. When you use the Logs Explorer page, you can select a different log scope, which causes the page to search the resources listed in the selected scope, and then refresh the display.

We recommend configuring the default log scope in the following scenarios:

  • You route log data to a centralized log bucket.
  • You route log data to other projects or to log buckets stored by another project.
  • You use log views.

For more information, see Create and manage log scopes.

Metrics scope

Configure the metrics scope to list all the projects which store metric data, so that your charts and alerting policies can display or monitor an aggregated view of your metric data.

Each Google Cloud project contains a single metrics scope, and this scope defaults to list only the project.

We recommend configuring the metrics scope when any of the following is true:

  • You want to chart data stored in different projects.
  • You want an alerting policy to monitor data stored in different projects.
  • You register applications with App Hub. For information about this scenario, see Metrics scopes for management projects.

For more information, see Metrics scopes overview.

Trace scopes

Configure the default trace scope to list all projects that store trace data, so that when you open the Trace Explorer page, you have an aggregated view of your trace data.

You can create multiple trace scopes. When you use the Trace Explorer page, you can select a different trace scope, which causes the page to search the projects listed in the selected scope, and then refresh the display.

We recommend configuring the trace scopes when you want a unified view of the trace data that is stored in different projects.

For more information, see Create and manage trace scopes.

Configure the observability scope

This section doesn't apply to folders or organizations.

For log and trace data, your Identity and Access Management (IAM) roles on the the project you are viewing, and any searched projects and log views, affect what data is returned by the query. If you issue a query to view log data that you don't have permission to view, then the query doesn't return any log data.

For metric data, when a project's metrics scope is configured, the project is granted read-access to the metric data stored by the projects listed in its metrics scope. When a user is granted an Identity and Access Management role that lets them view metric data in a project, they can view metric data available to the project.

Before you begin

  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

  2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  3. Verify that billing is enabled for your Google Cloud project.

  4. Enable the Observability API.

    Enable the API

  5. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  6. Verify that billing is enabled for your Google Cloud project.

  7. Enable the Observability API.

    Enable the API

    8.

  8. To get the permissions that you need to create and view scopes, ask your administrator to grant you the following IAM roles:

    • To create and view log scopes and to get the default log scope: Logs Configuration Writer (roles/logging.configWriter) on your project
    • To modify a metrics scopes: Monitoring Admin (roles/monitoring.admin) on your project and on each project you want to add to the metrics scopes
    • To create and view trace scopes, and to get and set default scopes: Observability Scopes Editor (roles/observability.scopesEditor) on your project

    For more information about granting roles, see Manage access to projects, folders, and organizations.

    You might also be able to get the required permissions through custom roles or other predefined roles.

    The Observability Scopes Editor role includes private permissions that let you create and view trace scopes. These permissions aren't available for inclusion in custom IAM roles.

  9. Select the tab for how you plan to use the samples on this page:

    Console

    When you use the Google Cloud console to access Google Cloud services and APIs, you don't need to set up authentication.

    gcloud

    In the Google Cloud console, activate Cloud Shell.

    Activate Cloud Shell

    At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

    REST

    To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.

      Install the Google Cloud CLI. After installation, initialize the Google Cloud CLI by running the following command: 

      gcloud init

      If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

    For more information, see Authenticate for using REST in the Google Cloud authentication documentation.

View and set the default scope

Console

To configure the observability scope, you configure its components, which are the default log scope, the metrics scope, and the default trace scope:

  1. In the Google Cloud console, go to the  Settings page:

    Go to Settings

    If you use the search bar to find this page, then select the result whose subheading is Monitoring.

  2. In the toolbar of the Google Cloud console, select your Google Cloud project. For App Hub configurations, select the App Hub host project or the app-enabled folder's management project.

  3. Configure the default log scope:

    1. Select the Log Scopes tab.

      Existing log scopes are listed. The entry with the "Default" icon, , is the default log scope. If you want to create a log scope, click Create log scope and then complete the dialog. For more information, see Create and manage log scopes.

    2. Find the entry that you want to designate as the default, click  More, and then select Set as default.

  4. Configure the metrics scope:

    1. Select the metrics scope tab.
    2. In the Google Cloud Projects pane, click Add Projects, and then complete the dialog. For more information, see Configure metrics scopes.

  5. Configure the default trace scope:

    1. Select the Trace Scopes tab and then do the following:

      Existing trace scopes are listed. The entry with the "Default" icon, , is the default trace scope. If you want to create a trace scope, click Create log scope and then complete the dialog. For more information, see Create and manage trace scopes.

    2. Find the entry that you want to designate as the default, click  More, and then select Set as default.

gcloud

To view and set the default observability scope, do the following:

  1. To view the settings for the default scope, run the gcloud observability scopes describe command.

    Before using any of the command data below, make the following replacements:

    • OBSERVABILITY_SCOPE_ID: The name of a Scope object. This value must be set to _Default.
    • LOCATION: The location field must be set to global.
    • PROJECT_ID: The identifier of the project.

    Execute the gcloud observability scopes describe command:

    Linux, macOS, or Cloud Shell

    gcloud observability scopes describe OBSERVABILITY_SCOPE_ID \    --location=LOCATION\    --project=PROJECT_ID

    Windows (PowerShell)

    gcloud observability scopes describe OBSERVABILITY_SCOPE_ID `    --location=LOCATION`    --project=PROJECT_ID

    Windows (cmd.exe)

    gcloud observability scopes describe OBSERVABILITY_SCOPE_ID ^    --location=LOCATION^    --project=PROJECT_ID

    The response to the command is similar to the following: 

     logScope: logging.googleapis.com/projects/my-project/locations/global/logScopes/_Default traceScope: projects/my-project/locations/global/traceScopes/_Default name: projects/my-project/locations/global/scopes/_Default

  2. To update the default scope, run the gcloud observability scopes update command.

    Before using any of the command data below, make the following replacements:

    • OBSERVABILITY_SCOPE_ID: The name of a Scope object. This value must be set to _Default.
    • LOG_SCOPE_FQN_ID: The fully-qualified ID of the log scope. This field has the following format: 
      logging.googleapis.com/projects/PROJECT_ID/locations/LOCATION/logScopes/LOG_SCOPE_ID

      In the previous expression, LOG_SCOPE_ID is the ID of the log scope. For example, my-scope.

    • LOCATION: The location field must be set to global.
    • PROJECT_ID: The identifier of the project.

    Execute the gcloud observability scopes update command:

    Linux, macOS, or Cloud Shell

    gcloud observability scopes update OBSERVABILITY_SCOPE_ID \    --log-scope=LOG_SCOPE_FQN_ID\    --location=LOCATION\    --project=PROJECT_ID

    Windows (PowerShell)

    gcloud observability scopes update OBSERVABILITY_SCOPE_ID `    --log-scope=LOG_SCOPE_FQN_ID`    --location=LOCATION`    --project=PROJECT_ID

    Windows (cmd.exe)

    gcloud observability scopes update OBSERVABILITY_SCOPE_ID ^    --log-scope=LOG_SCOPE_FQN_ID^    --location=LOCATION^    --project=PROJECT_ID

    For example, if the value of the LOG_SCOPE_ID is my-scope, then the response is similar to the following: 

     Updated scope [_Default]. logScope: logging.googleapis.com/projects/my-project/locations/global/logScopes/my-scope name: projects/my-project/locations/global/scopes/_Default

REST

To get and set the default log scope or the default trace scope by using an API call, you configure the observability scope. The observability scope lists the default log scope and the default trace scope:

  • To get the default observability scope for a project, send a request to the projects.locations.scopes.get endpoint. You must specify a path parameter. The response is a Scope object, which lists the default log scope and the default trace scope.

  • To update the default observability scope for a project, send a request to the projects.locations.scopes.patch endpoint. You must specify a path parameter, query parameters, and provide a Scope object. The query parameters identify which fields are changed. The response is a Scope object.

The path parameter for both endpoints has the following form:

projects/PROJECT_ID/locations/LOCATION/scopes/OBSERVABILITY_SCOPE_ID

The fields in the previous expression have the following meaning:

  • PROJECT_ID: The identifier of the project. For App Hub configurations, select the App Hub host project or the app-enabled folder's management project.
  • LOCATION: The location field must be set to global.
  • OBSERVABILITY_SCOPE_ID: The name of a Scope object. This field must be set to _Default. The Scope object with the name _Default, which is created automatically, stores information about the default log scope and the default trace scope.

To send a command to an API endpoint, you can use the APIs Explorer, which lets you issue a command from a reference page. For example, to get the current default scope, you can do the following:

  1. Click projects.locations.scopes.get.

  2. In the Try this method widget, enter the following in the name field:

    projects/PROJECT_ID/locations/global/scopes/_Default

    Before you copy the previous field, replace PROJECT_ID with the name of your project.

  3. Select Execute.

  4. In the authorization dialog, complete the required steps.

    The response is similar to the following:

    { "name": "projects/my-project/locations/global/scopes/_Default", "logScope": "logging.googleapis.com/projects/my-project/locations/global/logScopes/_Default" "traceScope": "projects/my-project/locations/global/traceScopes/_Default" }