Chapter 6. Adding a Google Cloud source to cost management

To add a Google Cloud account to cost management, you must configure your Google Cloud to provide metrics, then add your Google Cloud account as a source from the cloud.redhat.com user interface.

Note

You must have a Red Hat account user with Organization Administrator or Sources Administrator entitlements before you can add sources to cost management.

Before you can add your Google Cloud account to cost management as a data source, you must configure the following services on your Google Cloud account to allow cost management access to metrics:

  • cost management Google Cloud project.
  • Billing service account member with the correct role to export your data to cloud.redhat.com.
  • BigQuery dataset to contain the cost data.
  • Billing export that sends the cost data to your BigQuery dataset.

As you will complete some of the following steps in the Google Cloud console, and some steps in the cost management user interface, keep both applications open in a web browser.

Add your Google Cloud source to cost management from the settings area at https://cloud.redhat.com/settings/sources/.

Note

As non-Red Hat products and documentation can change without notice, instructions for configuring the third-party sources provided in this guide are general and correct at the time of publishing. See the Google Cloud Platform documentation for the most up-to-date and accurate information.

6.1. Creating a Google Cloud project

It is strongly recommended that you create a new Google Cloud project to gather and send your cost reports to cloud.redhat.com. This procedure outlines the steps required to create and name a new project from the Google Cloud Console.

Prerequisites

  • Access to Google Cloud Console with resourcemanager.projects.create permission

Procedure

  1. Navigate to IAM & AdminCreate a Project in Google Cloud Console.
  2. Enter a Project name in the new page that appears and select your billing account.
  3. Select the Organization.
  4. Enter the parent organization in the Location box.
  5. Click Create

Verification steps

  1. Navigate to the Google Cloud Console Dashboard
  2. Verify the project is in the top menu bar.

Additional resources

6.2. Creating a Google Cloud Identity and Access Management role

A custom Identity and Access Management (IAM) role for cost management gives access to specific cost related resources required to enable a Google Cloud Platform source and prohibits access to other resources.

Prerequisites

  • Access to Google Cloud Console with these permissions:

    • resourcemanager.projects.get
    • resourcemanager.projects.getIamPolicy
    • resourcemanager.projects.setIamPolicy
  • Google Cloud project

Procedure

  1. Navigate to IAM & AdminRoles in Google Cloud Console.
  2. Select the cost management project from the dropdown in the top menu bar.
  3. Click + Create role
  4. Enter a Title, Description and ID for the role.
  5. Click + ADD PERMISSIONS
  6. Use the Enter property name or value field to search and select these four permissions for your custom role:

    • bigquery.jobs.create
    • bigquery.tables.getData
    • bigquery.tables.get
    • bigquery.tables.list
  7. Click ADD
  8. Click CREATE

Verification steps

  1. Navigate to IAM & AdminRoles.
  2. Click the title of the custom role you created.
  3. Verify the role has these four assigned permissions:

    • bigquery.jobs.create
    • bigquery.tables.getData
    • bigquery.tables.get
    • bigquery.tables.list

Additional resources

6.3. Adding a billing service account member to your Google Cloud project

A new billing service account member must be created in your project that can export cost reports to cloud.redhat.com.

Prerequisites

  • Access to Google Cloud Console with these permissions:

    • resourcemanager.projects.get
    • resourcemanager.projects.getIamPolicy
    • resourcemanager.projects.setIamPolicy
  • Google Cloud project
  • cost management Identity and Access Management (IAM) role

Procedure

  1. Navigate to IAM & AdminIAM in Google Cloud Console.
  2. Select the cost management project from the dropdown in the top menu bar.
  3. Click ADD at the top of the page.
  4. Paste this into the New members field:

    billing-export@red-hat-cost-management.iam.gserviceaccount.com

  5. Select the cost management IAM role.
  6. Click SAVE

Verification steps

  1. Navigate to IAM & AdminIAM.
  2. Verify the new member is present with the correct role.

Additional resources

6.4. Creating a Google Cloud BigQuery dataset

Create a BigQuery dataset to collect and store the billing data for cost management.

Prerequisites

  • Access to Google Cloud Console with bigquery.datasets.create permission
  • Google Cloud project

Procedure

  1. Navigate to Big DataBigQuery in Google Cloud Console.
  2. Select the cost management project in the Explorer panel.
  3. Click CREATE DATASET
  4. Enter a name for your dataset in the Dataset ID field.
  5. Click CREATE DATASET

Verification steps

  1. Select the cost management project you created in the Explorer panel.
  2. Verify the dataset you created is nested under the cost management project.

6.5. Exporting Google Cloud billing data to BigQuery

Enabling a billing export to BigQuery sends your Google Cloud billing data (such as usage, cost estimates, and pricing data) automatically to the cost management BigQuery dataset.

Prerequisites

Procedure

  1. Navigate to BillingBilling export in Google Cloud Console.
  2. Click the BIGQUERY EXPORT tab.
  3. Click EDIT SETTINGS in the Daily cost detail section.
  4. Select the cost management Project and Billing export dataset you created in the dropdown menus.
  5. Click SAVE.

Verification steps

  1. Verify a green checkmark with Enabled in the Daily cost detail section, with correct Project name and Dataset name.

6.5.1. Viewing billing tables in BigQuery

You may want to review the metrics collected and sent to cost management. This can also assist with troubleshooting incorrect or missing data in cost management.

Note

Google may take several hours to export billing data to your BigQuery dataset.

Prerequisites

  • Access to Google Cloud console with bigquery.dataViewer role

Procedure

  1. Navigate to Big DataBigQuery in Google Cloud Console.
  2. Select the cost management project in the Explorer panel.
  3. Click gcp_billing_export_v1_xxxxxx_xxxxxx_xxxxxx table under the cost management dataset.
  4. Click the Preview tab to view the metrics.

6.6. Adding your Google Cloud account as a source

Using the information from the previous steps, you can now add your Google Cloud account as a source. After adding a Google Cloud source, the cost management application processes the cost and usage data from your Google Cloud account and makes it viewable.

Prerequisites

Procedure

  1. From cloud.redhat.com, click configuration gear (Settings).
  2. Click Sources
  3. Click Add source in the Cloud sources tab.
  4. Enter a name for the source and click Next
  5. In the dialog, select Google Cloud as the source type and cost management as the application.
  6. Click Next.
  7. Enter your Project ID and click Next.
  8. Click Next to verify you have created an Identity and Access Management (IAM) role.
  9. Click Next to verify you have created a new billing service account user and assigned the correct role.
  10. Enter your BigQuery Dataset ID and click Next
  11. Click Next to verify you have enabled your BigQuery billing export.
  12. Review the details and click Add

Verification steps

  • Verify your source is listed with Available status in the Sources page.

    Important

    Google may take several hours to gather and export billing data to cost management. In the meantime, you will receive a configuration in progress message, and your source status will display as Unknown in the Sources page.

Table 6.1. Troubleshooting

IssueCauseRemediation steps

Incorrect IAM permissions for project yourprojectID

The billing service account member does not have the correct role or permissions for cost management.

Make sure the billing service account has the cost management role with the correct permissions.

Unable to find dataset: yourdatasetID in project: yourprojectID

The BigQuery dataset ID was entered incorrectly or does not exist.

Make sure the BigQuery dataset ID matches the one created for cost management.

Source status "Unknown" longer than 24 hours

cost management cannot find cost data in the BigQuery dataset provided.

Make sure your billing export is configured and billing tables exist in your BigQuery dataset.