Chapter 30. Exporting 3scale API analytics beyond built-in capabilities

Create scripts that extend the capabilities of built-in 3scale analytics so that you can retrieve information that is not provided by default in the features.

By using Account Management and Analytics API (Enterprise only), you can create scripts to retrieve information that you need in your preferred format. The use case described here can be used to help you in your own scenario to get the data needed out of 3scale.

Reasons for customized scripts

3scale continuously improves the features available on your API Dashboard. However, you may be ahead of our development plans and have a very specific need not yet supported.

To satisfy the needs for API management, the 3scale Admin Portal provides the tools you need to access all your data. It takes resources to write the scripts, however customized scripts gives you total flexibility and control to implement various use cases.

30.1. Example of using 3scale to extract data about application use

A customer was in the process of onboarding thousands of new developers per week. This customer could tackle some aspects of the onboarding because 3scale provides automated necessities such as key provisioning, sign-up workflows, and email communication. There was, however, something that was not possible to do with 3scale, which was quite important for them.

Since the customer was onboarding many people, the company needed a straight-forward way to classify the new developers based on their engagement with APIs so that their operations and marketing teams could interact with the new developers more effectively. At least at the required level of detail, such a feature was not yet available in the built-in analytics tools provided by 3scale. However, it was possible to extract data using the 3scale Account and Analytics API because all the data was available in the system.

Example: Customer requirements

They would like to know how many new developers have signed up for the free evaluation plan in the last 10 days, split up different ways.

First, they wanted to know how many developers signed up but never used their API.

Second, they wanted to split the developers that had used their API into two groups:

  • Developers that used it for a period of time – say the first half of the 10 days – and then stopped using the API. These developers tried it out, but became inactive.
  • Developers that have been using the API consistently. For those, they would like to know the percent growth (or decline).

This information is available in 3scale built-in analytics. The problem is that there is no view to show it aggregated, which makes the whole experience quite cumbersome.

30.2. Extracting 3scale application analytics in custom procedures

To extract application analytics, start by working with ActiveDocs. 3scale ActiveDocs is available in your Admin Portal, under Account Settings > Integrate > 3scale API Docs. 3scale has all of its APIs available as ActiveDocs so that you can try them from your browser. This allows you to find the request that best serves your needs, get the request (curl-like) and get a grasp of the response. The following figure provides an ActiveDocs example:

DIY Analytics

This is the ActiveDocs for the API request that fetches all applications for which the script will extract analytics.

  • After you have done the research with the ActiveDocs, specify the request in your scripting language of choice. The example uses Ruby.
  • Repeat until you have a script that does exactly what you need. For the example of the extended analytics, the script is available as a gist. You can try it out in your account.

ActiveDocs lets you quickly understand what the API is able to do. Then, it is a matter of finding which 3 or 4 requests are needed for the task you want to do and putting a script together.

The following procedure shows the steps that achieved the custom analytics that the example customer wanted.

Procedure

  1. Retrieve the full list of applications. This operation requires pagination.

    def api_call_applications_list(domain, provider_key)
      done = false
      res = Array.new
      page = 1
    
          while !done
        url = "https://#{domain}/admin/api/applications.xml?provider_key=#{provider_key}&page=#{page}&per_page=100"
        page += 1
        response = RestClient.get url
        raise Exception.new("Wrong response code (#{response.code}) in request #{url}") if response.code!=200
        document = Nokogiri::XML(response.to_str)    done = document.xpath("applications/@current_page").text == document.xpath("applications/@total_pages").text
        document.xpath("//application").each do |item|
          app = Hash.new
          app["created_at"] = DateTime.parse(item.xpath("created_at").text)
          app["plan_name"] = item.xpath("plan/name").text
          app["service_id"] = item.xpath("plan/service_id").text
          app["account_id"] = item.xpath("user_account_id").text
          app["id"] = item.xpath("id").text
          res << app
        end
      end
      return res
    end
  2. Filter the applications that do not meet the criteria, i.e. the plan must be "evaluation" and they have to be newer than 10 days.

    def filter_applications(domain, provider_key, plan_name, num_of_days)
      res = api_call_applications_list(domain, provider_key)
      res.each do |item|
        res.delete(item) if item["plan_name"] != plan_name
        res.delete(item) if item["created_at"] > (DateTime.now - num_of_days)
      end
      return res
    end
  3. For each application that meets the criteria, get its usage, which is the number of hits the application has had in the last 10 days.

    def api_call_application_usage(domain, provider_key, application_id, metric, from, to, granularity)
      url = "https://#{domain}/stats/applications/#{application_id}/usage.xml?provider_key=#{provider_key}&metric_name=#{metric}&since=#{from}&until=#{to}&granularity=#{granularity}"
      response = RestClient.get url
      raise Exception.new("Wrong response code (#{response.code}) in request #{url}") if response.code!=200
      document = Nokogiri::XML(response.to_str)
      return document.xpath("//usage/data/values").text.split(",")
    end
  4. Cross-reference the applications to the accounts, because the information for the developers is stored in the account object.

    def api_call_account_read(domain, provider_key, account_id)
      url = "https://#{domain}/admin/api/accounts/#{account_id}.xml?provider_key=#{provider_key}"
      response = RestClient.get url
      raise Exception.new("Wrong response code (#{response.code}) in request #{url}") if response.code!=200
      document = Nokogiri::XML(response.to_str)
      account = Hash.new
      account["email"] = document.xpath("//users/user/email").text
      account["name"] = document.xpath("//users/user/first_name").text + " " + document.xpath("//users/user/last_name").text
      return account
    end
  5. Put everything together to complete the script. You have a script that gets the information that was not yet available in 3scale’s built-in analytics. You can also get the full script as a gist.