Adding a Google Cloud source to cost management

Cost Management Service 2022

Learn how to add and configure your Google Cloud sources

Red Hat Customer Content Services

Abstract

This guide describes how to add a Google CLoud source to cost management.

Chapter 1. Adding a Google Cloud source to cost management

To add a Google Cloud account to cost management, you must configure your Google Cloud to provide metrics, then add your Google Cloud account as a source from the console.redhat.com user interface.

Note

You must have a Red Hat account user with Sources Administrator entitlements before you can add sources to cost management.

Before you can add your Google Cloud account to cost management as a data source, you must configure the following services on your Google Cloud account to allow cost management access to metrics:

  • Cost management Google Cloud project.
  • Billing service account member with the correct role to export your data to console.redhat.com.
  • BigQuery dataset to contain the cost data.
  • Billing export that sends the cost data to your BigQuery dataset.

As you will complete some of the following steps in the Google Cloud console, and some steps in the cost management user interface, keep both applications open in a web browser.

Add your Google Cloud source to cost management from the settings area at https://console.redhat.com/settings/sources.

Note

As non-Red Hat products and documentation can change without notice, instructions for configuring the third-party sources provided in this guide are general and correct at the time of publishing. See the Google Cloud Platform documentation for the most up-to-date and accurate information.

1.1. Creating a Google Cloud project

It is strongly recommended that you create a new Google Cloud project to gather and send your cost reports to console.redhat.com. This procedure outlines the steps required to create and name a new project from the Google Cloud Console.

Prerequisites

  • Access to Google Cloud Console with resourcemanager.projects.create permission

Procedure

  1. Navigate to IAM & AdminCreate a Project in Google Cloud Console.
  2. Enter a Project name in the new page that appears and select your billing account.
  3. Select the Organization.
  4. Enter the parent organization in the Location box.
  5. Click Create

Verification steps

  1. Navigate to the Google Cloud Console Dashboard
  2. Verify the project is in the top menu bar.

Additional resources

1.2. Creating a Google Cloud Identity and Access Management role

A custom Identity and Access Management (IAM) role for cost management gives access to specific cost related resources required to enable a Google Cloud Platform source and prohibits access to other resources.

Prerequisites

  • Access to Google Cloud Console with these permissions:

    • resourcemanager.projects.get
    • resourcemanager.projects.getIamPolicy
    • resourcemanager.projects.setIamPolicy
  • Google Cloud project

Procedure

  1. Navigate to IAM & AdminRoles in Google Cloud Console.
  2. Select the cost management project from the dropdown in the top menu bar.
  3. Click + Create role
  4. Enter a Title, Description and ID for the role.
  5. Click + ADD PERMISSIONS
  6. Use the Enter property name or value field to search and select these four permissions for your custom role:

    • bigquery.jobs.create
    • bigquery.tables.getData
    • bigquery.tables.get
    • bigquery.tables.list
  7. Click ADD
  8. Click CREATE

Verification steps

  1. Navigate to IAM & AdminRoles.
  2. Click the title of the custom role you created.
  3. Verify the role has these four assigned permissions:

    • bigquery.jobs.create
    • bigquery.tables.getData
    • bigquery.tables.get
    • bigquery.tables.list

Additional resources

1.3. Adding a billing service account member to your Google Cloud project

A new billing service account member must be created in your project that can export cost reports to console.redhat.com.

Prerequisites

  • Access to Google Cloud Console with these permissions:

    • resourcemanager.projects.get
    • resourcemanager.projects.getIamPolicy
    • resourcemanager.projects.setIamPolicy
  • Google Cloud project
  • cost management Identity and Access Management (IAM) role

Procedure

  1. Navigate to IAM & AdminIAM in Google Cloud Console.
  2. Select the cost management project from the dropdown in the top menu bar.
  3. Click ADD at the top of the page.
  4. Paste this into the New members field:

    billing-export@red-hat-cost-management.iam.gserviceaccount.com

  5. Select the cost management IAM role.
  6. Click SAVE

Verification steps

  1. Navigate to IAM & AdminIAM.
  2. Verify the new member is present with the correct role.

Additional resources

1.4. Creating a Google Cloud BigQuery dataset

Create a BigQuery dataset to collect and store the billing data for cost management.

Prerequisites

  • Access to Google Cloud Console with bigquery.datasets.create permission
  • Google Cloud project

Procedure

  1. Navigate to Big DataBigQuery in Google Cloud Console.
  2. Select the cost management project in the Explorer panel.
  3. Click CREATE DATASET
  4. Enter a name for your dataset in the Dataset ID field.
  5. Click CREATE DATASET

Verification steps

  1. Select the cost management project you created in the Explorer panel.
  2. Verify the dataset you created is nested under the cost management project.

1.5. Exporting Google Cloud billing data to BigQuery

Enabling a billing export to BigQuery sends your Google Cloud billing data (such as usage, cost estimates, and pricing data) automatically to the cost management BigQuery dataset.

Prerequisites

Procedure

  1. Navigate to BillingBilling export in Google Cloud Console.
  2. Click the BIGQUERY EXPORT tab.
  3. Click EDIT SETTINGS in the Daily cost detail section.
  4. Select the cost management Project and Billing export dataset you created in the dropdown menus.
  5. Click SAVE.

Verification steps

  1. Verify a green checkmark with Enabled in the Daily cost detail section, with correct Project name and Dataset name.

1.5.1. Viewing billing tables in BigQuery

You may want to review the metrics collected and sent to cost management. This can also assist with troubleshooting incorrect or missing data in cost management.

Note

Google may take several hours to export billing data to your BigQuery dataset.

Prerequisites

  • Access to Google Cloud console with bigquery.dataViewer role

Procedure

  1. Navigate to Big DataBigQuery in Google Cloud Console.
  2. Select the cost management project in the Explorer panel.
  3. Click gcp_billing_export_v1_xxxxxx_xxxxxx_xxxxxx table under the cost management dataset.
  4. Click the Preview tab to view the metrics.

1.6. Adding your Google Cloud account as a source

Using the information from the previous steps, you can now add your Google Cloud account as a source. After adding a Google Cloud source, the cost management application processes the cost and usage data from your Google Cloud account and makes it viewable.

Prerequisites

Procedure

  1. From console.redhat.com, click configuration gear (Settings).
  2. Click Sources
  3. Click Add source in the Cloud sources tab.
  4. Enter a name for the source and click Next
  5. In the dialog, select Google Cloud as the source type and cost management as the application.
  6. Click Next.
  7. Enter your Project ID and click Next.
  8. Click Next to verify you have created an Identity and Access Management (IAM) role.
  9. Click Next to verify you have created a new billing service account user and assigned the correct role.
  10. Enter your BigQuery Dataset ID and click Next
  11. Click Next to verify you have enabled your BigQuery billing export.
  12. Review the details and click Add

Verification steps

  • Verify your source is listed with Available status in the Sources page.

    Important

    Google may take several hours to gather and export billing data to cost management. In the meantime, you will receive a configuration in progress message, and your source status will display as Unknown in the Sources page.

Table 1.1. Troubleshooting

IssueCauseRemediation steps

Incorrect IAM permissions for project yourprojectID.

The billing service account member does not have the correct role or permissions for cost management.

Make sure the billing service account has the cost management role with the correct permissions.

Unable to find dataset: yourdatasetID in project: yourprojectID.

The BigQuery dataset ID was entered incorrectly or does not exist.

Make sure the BigQuery dataset ID matches the one created for cost management.

Source status "Unknown" longer than 24 hours.

Cost management cannot find cost data in the BigQuery dataset provided.

Make sure your billing export is configured and billing tables exist in your BigQuery dataset.

Chapter 2. Next steps for managing your costs

After adding your OpenShift Container Platform and Google Cloud sources, on the cost management Overview page (console.redhat.com/openshift/cost-management/), your cost data will be sorted into OpenShift and Infrastructure tabs. From here, you can use Perspective to select different views of your cost data.

You can also use the left navigation menu to view the additional details of your costs by service.

2.1. Limit access to cost management resources

After adding and configuring sources in cost management, it is a good idea to limit access to cost data and resources. For example, you may not want users to have access to all cost data, but instead only data specific to their projects or organization. Using role-based access control, you can limit the visibility of resources involved in cost management reports. For example, you may want to restrict a user’s view to only AWS sources, instead of the entire environment.

For more information about limiting access see, Limiting access to cost management resources.

2.2. Configure tagging for your sources

The cost management application tracks cloud and infrastructure costs using tags (called labels in OpenShift), which you can refine to filter and attribute to resources. Tags in cost management allow you to organize your resources by cost and to allocate the costs to different parts of your cloud infrastructure.

Important

Tags and labels can only be configured directly on a source. While you can choose what tags are activated in cost management, you cannot edit tags and labels in the cost management application.

See Managing cost data using tagging to learn more about:

  • Planning your tagging strategy to organize your view of cost data.
  • Understanding how cost management associates tags.
  • Configuring tags and labels on your sources.

2.3. Configure cost models to accurately report costs

Now that you have configured your sources to collect cost and usage data into cost management, you can configure cost models to associate prices to metrics and usage, and fine-tune the costs of running your cloud.

A cost model is a framework used to define the calculations for the costs stored in cost management, using raw costs and metrics. Costs generated by a cost model can be recorded, categorized and distributed to specific customers, business units or projects.

From the Cost Models area of cost management, you can:

  • Classify your costs as infrastructure or supplementary costs.
  • Capture monthly costs for OpenShift nodes and clusters.
  • Apply a markup to account for additional support costs.
  • Learn how to configure a cost model in Using cost models.

2.4. Use the Cost Explorer to visualize your costs

The cost management Cost Explorer allows you to create custom graphs of time-scaled cost and usage information to better visualize and interpret your costs.

See Visualizing your costs using Cost Explorer to learn more about:

  • Using Cost Explorer to identify abnormal events.
  • Understanding how your cost data changes over time.
  • Creating custom bar charts of your cost and usage data.
  • Exporting custom cost data tables.

Legal Notice

Copyright © 2022 Red Hat, Inc.
The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/. In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version.
Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, Red Hat Enterprise Linux, the Shadowman logo, the Red Hat logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
Java® is a registered trademark of Oracle and/or its affiliates.
XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.
MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
Node.js® is an official trademark of Joyent. Red Hat is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.
The OpenStack® Word Mark and OpenStack logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation's permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community.
All other trademarks are the property of their respective owners.