Integrating OpenShift Container Platform data into cost management

Cost Management Service 1-latest

Learn how to add and configure your OpenShift Container Platform integrations

Red Hat Customer Content Services

Abstract

Learn how to add an OpenShift Container Platform integration to cost management. Cost management is part of the Red Hat Insights portfolio of services. The Red Hat Insights suite of advanced analytical tools helps you to identify and prioritize impacts on your operations, security, and business.

Chapter 1. Integrating OpenShift Container Platform data into cost management

1.1. Installation tasks summary

Whether you are replacing a prior cost management operator with the Cost Management Metrics Operator or installing it for the first time, the basic tasks are the same.

Operator installation, configuration, and integration management can all be performed from the OpenShift Container Platform web console.

You will perform the following tasks to install the Cost Management Metrics Operator and begin using the cost management application in OpenShift Container Platform.

Note

To install and configure Cost Management Metrics Operator from the OpenShift Container Platform web console, you must use an account with cluster administrator privileges.

Prerequisites

Task summary

  • Install the Cost Management Metrics Operator (costmanagement-metrics-operator) and use the default token authentication.
  • Create a CostManagementMetricsConfig YAML file that configures costmanagement-metrics-operator
  • Create a cost management OpenShift Container Platform integration with a new installation, or confirm an existing integration with a replacement installation.
  • After installing the Cost Management Metrics Operator, delete the old cost operator. This task is required only when a previous cost management operator is installed.

If you use Basic authentication, additional steps are required to configure the Secret that holds username and password credentials.

1.2. Installing the cost operator

Learn how to install the Cost Management Metrics Operator from the OpenShift Container Platform web console.

Prerequisites

  • You logged in to the OpenShift Container Platform web console and have cluster administrator privileges.

Procedure

  1. Log in to the OpenShift Container Platform web console and click Operators > OperatorHub.
  2. Click Cost Management Metrics Operator.
  3. When the Install Operator window appears, select the costmanagement-metrics-operator namespace. If the namespace does not exist, we create it for you.
  4. Click Install. After a short wait, Cost Management Metrics Operator appears in the Installed Operators tab under Project: all projects or Project: costmanagement-metrics-operator.
Important

If a proxy with a custom CA certificate is configured, additional configuration is required to inject this certificate into Cost Management Metrics Operator. For more details, see Injecting a custom CA certificate in the OpenShift Container Platform documentation.

1.3. Configuring the operator instance for a new installation

After you install the costmanagement-metrics-operator instance, you can configure it in the OpenShift Container Platform web console.

Prerequisites

  • You logged in to the OpenShift Container Platform web console and have cluster administrator privileges.
  • Cost Management Metrics Operator appears in the Installed Operators tab.

Procedure

  1. From Name in the list of installed operators, click Cost Management Metrics Operator. An Installed Operators > Operator Details window appears.
  2. From Details, click + Create Instance. An Cost Management Metrics Operator > Create CostManagementMetricsConfig window appears.
  3. Select YAML view to view and modify the contents of the YAML configuration file.
  4. Create a cost management instance for the Cost Management Metrics Operator by editing the following two lines in the YAML file:

        create_source: false
        name: ''
    1. Change false to true.
    2. Change '' to the name of your integration. If a name is not provided, the operator defaults to using the cluster ID as the name when creating the integration.

      Example

          create_source: true
          name: my-openshift-cost-source

  5. Click Create.

1.4. Replacing an operator

Learn how to replace an existing cost management operator with the Cost Management Metrics Operator by editing your YAML configuration file.

Prerequisites

Procedure

  1. From the Name heading in the list of installed operators, click Cost Management Metrics Operator. An Installed Operators > Operator Details window appears.
  2. From Details, click + Create Instance. An Cost Management Metrics Operator > Create CostManagementMetricsConfig window appears.
  3. Click YAML view to view and edit the contents of the CostManagementMetricsConfig YAML file.
  4. In another tab, open Red Hat Hybrid Cloud Console and log in with your Organization Administrator account:

    1. Click Settings Settings icon .
    2. Click the Integrations tab to display existing integrations.
    3. Select an existing cost management integration and copy its name.
  5. Return to the CostManagementMetricsConfig YAML file. Replace the empty string in name: '' with the integration name that you copied from cost management.

        create_source: false
        name: ''    <<<< replace this string

    Do not edit create_source: false.

  6. Click Create.
Important

When you replace an existing cost management operator with the Cost Management Metrics Operator, and you want to use an existing integration, you must make sure that name: '' in the YAML file exactly matches your existing integration.

1.5. Removing a cost operator

After you install the costmanagement-metrics-operator, you must uninstall the existing cost management operator.

To avoid disruption in your cost management reports, you can wait 24 to 48 hours before you remove the existing operator to verify that costmanagement-metrics-operator is working correctly.

Prerequisites

  • You installed a cost management operator.
  • You installed the Cost Management Metrics Operator.
  • You logged in to the OpenShift Container Platform web console and have cluster administrator privileges.
  • You can view the operators in the Installed Operators tab.

Procedure

  1. From Installed Operators, select the operator that you want to remove.
  2. Click Options more options in the operator’s row.
  3. Click Uninstall Operator and then click Confirm.
  4. In a new tab, open the OpenShift Container Platform web console:

    1. Go to Administration > Custom Resource Definitions.
    2. In the window that displays the custom resource definitions (CRD), locate the CostManagement CRD and the CostManagementData CRD for cost-mgmt-operator or the KokuMetricsConfig CRD for koku-metrics-operator.
    3. For each CRD, click Options > Delete Custom Resource Definition. Click Confirm.

After you delete all of the CRDs, the operator is fully uninstalled.

1.6. Troubleshooting issues with your cost operator

To verify that the cost management operator is functioning correctly, check that your YAML file is properly configured.

Prerequisites

Procedure

  1. Click the Installed Operators tab.
  2. In the list of installed operators, click Cost Management Metrics Operator.A metrics operator window opens.
  3. Click the CostManagementMetricsConfig tab to show a list of the configuration file names.
  4. In the file name list, click the configuration file that you want to verify. In the default installation, the file name is costmanagementmetricscfg-sample. A Details window opens.
  5. Click YAML and check the following items:

    • prometheus_configured and prometheus_connected should be set to true:

        prometheus:
          last_query_start_time: '2021-01-25T20:59:06Z'
          last_query_success_time: '2021-01-25T20:59:06Z'
          prometheus_configured: true
          prometheus_connected: true
          service_address: 'https://thanos-querier.openshift-monitoring.svc:9091'
          skip_tls_verification: false
    • ingress_path, last_successful_upload_time, last_upload_status, and last_upload_time should all have content:

       upload:
          ingress_path: /api/ingress/v1/upload
          last_successful_upload_time: '2021-01-25T20:59:35Z'
          last_upload_status: 202 Accepted
          last_upload_time: '2021-01-25T20:59:35Z'
          upload: true
          upload_cycle: 360
          upload_wait: 28
          validate_cert: true
Note

To collect data, cost management uses Prometheus queries that you can find in the source code.

1.7. Configuring basic authentication for cost operator

You can configure the cost operator to use basic authentication. By default, the cost operator uses token authentication.

There are two procedures required when you configure basic authentication.

1.7.1. Creating the secret key/value pair for basic authentication

Prerequisites

  • You are logged into the OpenShift Container Platform web console with cluster administrator privileges.
  • The Cost Management Metrics Operator appears in the Installed Operators tab.
  • You have a username and password for your Red Hat Hybrid Cloud Console account.

Procedure

This procedure describes setting up basic authentication using the OpenShift Container Platform web console.

  1. In the OpenShift Container Platform web console, click on the Workloads > Secrets tab.
  2. In the Secrets window, select Project:costmanagement-metrics-operator from the dropdown.
  3. Click the Create > Key/Value Secret selection.
  4. In the Create Key/Value Secret window enter the following information to create a new secret that contains a username key and a password key and a value for each key.

    1. Enter a name for your secret in the Secret Name field.

      basic-auth-secret
    2. In the Key field, enter username.

      username
    3. In the Value field for the key username, enter the actual username for your authorized Red Hat Hybrid Cloud Console user account.

      Value for username key

      your_red_hat_username

    4. Click the Add Key/Value link to add the required password key name and value.
    5. In the Key field, enter password.

      password
    6. In the Value field for the key password, enter the actual password for your authorized Red Hat Hybrid Cloud Console user account.

      Value for password key

      your_red_hat_password

    7. Click the Create button to complete the creation of your basic authorization secret.
    8. After you click the Create button, you can verify the key/value details for the secret.

      Note

      Do not add the secret to the workload.

1.7.2. Modifying the YAML file

Modify the Cost Management Metrics Operator API YAML file to use basic authentication with a secret username and password key/value pair.

Prerequisites

  • You are logged into the OpenShift Container Platform web console with cluster administrator privileges.
  • You created a secret name for the username and password key/value pair.
  • The Cost Management Metrics Operator is installed.

Procedure

  1. Click on the Operators > Installed Operators tab.
  2. Locate the row that contains Cost Management Metrics Operator and click on the Cost Management Metrics Operator link that is under the Provided APIs heading.
  3. When the CostManagementMetricsConfig window appears, click on the configuration file listed in the Name column.

    The default name is costmanagementmetricscfg-sample.

  4. When the costmanagementmetricscfg-sample window appears, click in the YAML tab to open an edit and view window.
  5. Locate the following lines in the YAML view.

      authentication:
        type: token
  6. Change type: token to type: basic.
  7. Insert a new line for secret_name. Enter the value for secret_name, which is the name you previously created.

    Example

      authentication:
        secret_name: basic-auth-secret
        type: basic

  8. Click the Save button. A confirmation message appears.

1.8. Configuring service account authentication for the cost operator

To configure service account authentication, complete the following two tasks:

Important

Service account authentication is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

1.8.1. Creating the secret key/value pair for service account authentication

Prerequisites

  • You are logged into the OpenShift Container Platform web console and have cluster administrator privileges.
  • The Cost Management Metrics Operator appears in the Installed Operators tab.
  • You have a client_id and client_secret for your Red Hat Hybrid Cloud Console account.

Procedure

The following procedure outlines how to set up service account authentication with the OpenShift Container Platform web console:

  1. In the OpenShift Container Platform web console, click Workloads > Secrets.
  2. In the Secrets window, select Project:costmanagement-metrics-operator from the drop-down.
  3. Click Create > Key/Value Secret.
  4. To create a new secret with a client_id key and a client_secret key, enter the following information in the Create Key/Value Secret window:

    1. In Secret Name, enter a name for your secret:

      service-account-auth-secret
    2. In Key, enter client_id.

      client_id
    3. In the Value field for the key client_id, enter the Client ID for your authorized Red Hat Hybrid Cloud Console user account:

      red_hat_service_account_client_id
    4. Click Add Key/Value to add the client_secret for the key name and value.
    5. In Key, enter client_secret:

      client_secret
    6. In the Value field for the key client_secret, enter the Client secret for your authorized Red Hat Hybrid Cloud Console user account:

      red_hat_service_account_client_secret
    7. Click Create to complete the creation of your service account authorization secret.
    8. Verify that the key/value details for the secret are correct.

      Note

      Do not add the secret to the workload.

1.8.2. Modifying the YAML file

To use authentication with a Client ID and Client secret key/value pair, you must modify the Cost Management Metrics Operator API YAML file.

Prerequisites

  • You are logged into the OpenShift Container Platform web console and have cluster administrator privileges.
  • You created a secret name for the Client ID and Client secret key/value pair.
  • The Cost Management Metrics Operator is installed.

Procedure

  1. Click Operators > Installed Operators.
  2. Locate the row that contains Cost Management Metrics Operator and click the Cost Management Metrics Operator link that is under the Provided APIs heading.
  3. When the CostManagementMetricsConfig window appears, click the configuration file in Name.

    The default name is costmanagementmetricscfg-sample.

  4. When the costmanagementmetricscfg-sample window appears, click the YAML tab to edit and view the content.
  5. Locate the following lines in the YAML window that you opened in the previous step:

     authentication:
       type: token
  6. Change type: token to type: service-account.
  7. Insert a new line for secret_name. Enter the value for secret_name, which is the name you previously created.

    Example

     authentication:
       secret_name: service-account-auth-secret
       type: service-account

  8. Click Save. A confirmation message appears.

1.9. Manually creating an Openshift Container Platform integration

You can automatically create your OpenShift Container Platform integration by following the steps in Installing a cost operator. However, some situations, such as restricted network installations, require that you create an OpenShift Container Platform integration manually on Red Hat Hybrid Cloud Console.

Prerequisites

Procedure

  1. From cost management, click Settings . The Integrations page should open.
  2. Click the Red Hat tab.
  3. Click Add integration, which opens the Add a cloud integration wizard.
  4. In Select your integration type, click Red Hat OpenShift Container Platform.
  5. In Application, click cost management. Then click Next.
  6. Enter a name in Integration name. Then click Next.
  7. In a new tab, access the OpenShift Container Platform web console. Go to Home > Overview and copy your Cluster Identifier.
  8. Back in cost management, enter your Cluster Identifier. Then click Next.
  9. Review the details and click Add to create the integration.

1.10. Adding a restricted network integration

You can install OpenShift Container Platform on a restricted network that does not have access to the internet.

The procedure to add an OpenShift Container Platform cluster operating on a restricted network as a cost management integration is different in the following ways:

  1. Operator Lifecycle Manager is configured to install and run local integrations.
  2. The costmanagement-metrics-operator is configured to store cost report CSV files locally using a persistent volume claim (PVC).
  3. Cost reports stored in the PVC are downloaded to a workstation.
  4. An OpenShift Container Platform integration is created manually.
  5. Cost reports are uploaded to Red Hat Hybrid Cloud Console from your workstation.

1.10.1. Installing the cost management operator on a restricted network

Operator Lifecycle Manager (OLM) cannot access the OpenShift Container Platform clusters that are installed on restricted networks because remote integrations require full Internet connectivity. In these situations, you must install and configure OLM to run locally.

Prerequisites

  • You installed an OpenShift Container Platform cluster.
  • You have a workstation with unrestricted network access.
  • You logged in to the OpenShift Container Platform web console and have cluster administrator privileges.

Procedure

  1. Complete the following OpenShift Container Platform procedure to create a local mirror of the costmanagement-metrics-operator: Using Operator Lifecycle Manager on restricted networks.

    Note

    The costmanagement-metrics-operator is in the redhat-operators catalog in the registry.redhat.io/redhat/redhat-operator-index:v4.11 index.

    Prune unwanted objects from the index before you push to the mirrored registry, but do not delete the costmanagement-metrics-operator package.

  2. Log in to the OpenShift Container Platform web console and click Operators > OperatorHub.
  3. Click Cost Management Metrics Operator.
  4. The Install Operator window should open. Select the costmanagement-metrics-operator namespace that you want to install. If the namespace does not exist, it gets created.
  5. Click Install.

Verification steps

  • After a short wait, Cost Management Metrics Operator appears in the Installed Operators tab in Project: all projects or Project: costmanagement-metrics-operator.

Additional resources

1.10.2. Configuring the Cost Operator on a restricted network

Learn how to run the costmanagement-metrics-operator on a restricted network.

Prerequisites

Procedure

  1. From the OpenShift Container Platform web console, select Operators > Installed Operators > costmanagement-metrics-operator > CostManagementMetricsConfig > Create Instance.
  2. Set a storage amount. If you do not specify an amount, the operator creates a default persistent volume claim (PVC) called costmanagement-metrics-operator-data with 10Gi of storage.

    Note

    To configure the costmanagement-metrics-operator to use a different PVC, edit volume_claim_template in YAML view.

  3. Select YAML view.
  4. Enter a value in max_reports_to_store to set the maximum number of reports that you want to store.
  5. Enter a value in upload_cycle to set how many minutes you want to pass between each report generation.

        packaging:
          max_reports_to_store: 30
          max_size_MB: 100
        upload:
          upload_cycle: 360
    Important

    The costmanagement-metrics-operator creates one report every 360 minutes by default. The default value of 30 reports and 360 minutes gives you 7.5 days of reports.

    After the maximum number of reports generate, any subsequent reports replace the oldest report in storage. To avoid losing reports, download them from your PVC.

  6. Set upload_toggle to false:

        upload:
          upload_cycle: 360
          upload_toggle: false
  7. Set source to empty braces:

        source: {}
  8. Set authentication to empty braces:

        authentication: {}
  9. Click Create.

Verification steps

  1. Select the CostManagementMetricsConfig that you created.
  2. Click YAML view.
  3. Verify that a report was created by viewing the data in packaging:

        packaging:
          last_successful_packaging_time: `current date and time`
          max_reports_to_store: 30
          max_size_MB: 100
          number_of_reports_stored: 1
          packaged_files:
            - >-
                /tmp/costmanagement-metrics-operator-reports/upload/YYYYMMDDTHHMMSS-cost-mgmt.tar.gz
    Note

    After configuration, costmanagement-metrics-operator generates an initial report. These reports are in packaged_files.

1.10.3. Downloading cost reports

If you configured the costmanagement-metrics-operator to run on a restricted network, the reports from the persistent volume claims (PVC) are temporarily stored in a workstation. Copy the reports to an unrestricted network.

The default configuration saves one week of reports. To avoid losing metrics data, download the reports locally and upload them to Red Hat Hybrid Cloud Console weekly.

You can configure any PVC, but by default, most PVCs are ReadWriteOnce. For ReadWriteOnce PVCs, the volume-shell must be attached to the same node as the operator pod.

Prerequisites

  • You have a workstation with unrestricted network access.
  • costmanagement-metrics-operator reports in your PVC.

Procedure

  1. Create the following pod and set claimName to the PVC with the report data:

    kind: Pod
    apiVersion: v1
    metadata:
      name: volume-shell
      namespace: costmanagement-metrics-operator
    spec:
      volumes:
      - name: costmanagement-metrics-operator-reports
        persistentVolumeClaim:
          claimName: costmanagement-metrics-operator-data
      containers:
      - name: volume-shell
        image: busybox
        command: ['sleep', '3600']
        volumeMounts:
        - name: costmanagement-metrics-operator-reports
          mountPath: /tmp/costmanagement-metrics-operator-reports
  2. Run rsync to copy all of the files from the PVC to a local folder:

    $ oc rsync volume-shell:/tmp/costmanagement-metrics-operator-reports/upload local/path/to/save/folder
  3. Confirm that the files were copied.
  4. Run the following command to connect to the pod and delete the contents of the upload folder:

    $ oc rsh volume-shell
    $ rm /tmp/costmanagement-metrics-operator-reports/upload/*
  5. (Optional) Run the following command to delete the pod that you used to connect to the PVC:

    $ oc delete -f volume-shell.yaml

Viewing your PVC usage

In the OpenShift tab in Red Hat Hybrid Cloud Console, your PVCs with the highest usage automatically populate under Persistent Volume Claims. To view all PVCs, click more at the end of the section.

You can filter your PVC data by the following fields: * Persistent volume claim * Cluster * StorageClass

Additional resources

1.10.4. Uploading cost reports to console.redhat.com

You must manually upload locally stored cost reports from a restricted network to Red Hat Hybrid Cloud Console.

Note

The default configuration saves one week of reports. Download the reports locally and upload them to Red Hat Hybrid Cloud Console weekly to avoid losing metrics data.

Prerequisites

Procedure

To upload your reports to Red Hat Hybrid Cloud Console, make the following edit:

  • Set USERNAME and PASSWORD to your Red Hat Hybrid Cloud Console login credentials and set FILE_NAME to the report that you want to upload:

    $ curl -vvvv -F "file=@FILE_NAME.tar.gz;type=application/vnd.redhat.hccm.tar+tgz"  https://cloud.redhat.com/api/ingress/v1/upload -u USERNAME:PASS

Verification steps

  1. From cost management, click OpenShift.
  2. On the OpenShift details page, confirm that you have OpenShift usage data for your cluster.

Chapter 2. Next steps for managing your costs

After adding your OpenShift Container Platform and cloud infrastructure integrations, in addition to showing cost data by integration, cost management will automatically show AWS and Microsoft Azure cost and usage related to running your OpenShift Container Platform clusters on their platforms.

On the cost management Overview page, your cost data is sorted into OpenShift and Infrastructure tabs. Select Perspective to toggle through different views of your cost data.

You can also use the global navigation menu to view additional details about your costs by cloud provider.

2.1. Limiting access to cost management resources

After you add and configure integrations in cost management, you can limit access to cost data and resources.

You might not want users to have access to all of your cost data. Instead, you can grant users access only to data that is specific to their projects or organizations. With role-based access control, you can limit the visibility of resources in cost management reports. For example, you can restrict a user’s view to only AWS integrations, rather than the entire environment.

To learn how to limit access, see the more in-depth guide Limiting access to cost management resources.

2.2. Configuring tagging for your integrations

The cost management application tracks cloud and infrastructure costs with tags. Tags are also known as labels in OpenShift.

You can refine tags in cost management to filter and attribute resources, organize your resources by cost, and allocate costs to different parts of your cloud infrastructure.

Important

You can only configure tags and labels directly on an integration. You can choose the tags that you activate in cost management, however, you cannot edit tags and labels in the cost management application.

To learn more about the following topics, see Managing cost data using tagging:

  • Planning your tagging strategy to organize your view of cost data
  • Understanding how cost management associates tags
  • Configuring tags and labels on your integrations

2.3. Configuring cost models to accurately report costs

Now that you configured your integrations to collect cost and usage data in cost management, you can configure cost models to associate prices to metrics and usage.

A cost model is a framework that uses raw costs and metrics to define calculations for the costs in cost management. You can record, categorize, and distribute the costs that the cost model generates to specific customers, business units, or projects.

In Cost Models, you can complete the following tasks:

  • Classifying your costs as infrastructure or supplementary costs
  • Capturing monthly costs for OpenShift nodes and clusters
  • Applying a markup to account for additional support costs

To learn how to configure a cost model, see Using cost models.

2.4. Visualizing your costs with Cost Explorer

Use cost management Cost Explorer to create custom graphs of time-scaled cost and usage information and ultimately better visualize and interpret your costs.

To learn more about the following topics, see Visualizing your costs using Cost Explorer:

  • Using Cost Explorer to identify abnormal events
  • Understanding how your cost data changes over time
  • Creating custom bar charts of your cost and usage data
  • Exporting custom cost data tables

Providing feedback on Red Hat documentation

If you found an error or have a suggestion on how to improve these guidelines, open an issue in the cost management Jira board and add the Documentation label.

We appreciate your feedback!

Legal Notice

Copyright © 2024 Red Hat, Inc.
The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/. In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version.
Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.
Red Hat, Red Hat Enterprise Linux, the Shadowman logo, the Red Hat logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.
Linux® is the registered trademark of Linus Torvalds in the United States and other countries.
Java® is a registered trademark of Oracle and/or its affiliates.
XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.
MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.
Node.js® is an official trademark of Joyent. Red Hat is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.
The OpenStack® Word Mark and OpenStack logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation's permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community.
All other trademarks are the property of their respective owners.