Ansible Automation Platform (AAP) External Logging via AWS

Updated -

Overview

This guide outlines the standard configuration for forwarding automation logs from Ansible Automation Platform (AAP) to external log aggregators using Amazon Web Services (AWS).

While AAP can log directly to many platforms, this article demonstrates an end-to-end example using a Serverless Architecture (Amazon API Gateway + Kinesis Firehose). This is a recommended pattern for AWS environments as it eliminates the need to manage a dedicated "Log Collector" VM, OS firewalls, or patching, while providing a secure HTTPS endpoint for AAP to transmit data to.

Step 1: Infrastructure Setup (AWS)

A. Create the Data Stream (Kinesis Firehose)

This service acts as the pipeline that receives logs and writes them to storage.

  1. Navigate to Kinesis in the AWS Console.
  2. Select Create Delivery Stream.
  3. Source: Direct PUT.
  4. Destination: Amazon S3.
  5. S3 Bucket: Create a new bucket (e.g., aap-automation-logs-2026) or select an existing one.
  6. Buffering: Set to 5 MB or 300 seconds (Standard recommended settings).
  7. Click Create Delivery Stream.

B. Create Security Roles (IAM)

Critical Step: API Gateway cannot write to Firehose without permission.

1. Create the API Gateway Role

  • Navigate to IAM > Roles > Create Role.
  • Trusted Entity: Select API Gateway.
  • Permissions: Create a new inline policy with the following JSON:
{
"Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "firehose:PutRecord",
      "Resource": "arn:aws:firehose:us-east-1:123456789012:deliverystream/YOUR_STREAM_NAME"
    }
  ]
}
  • Role Name: APIGateway-Firehose-Proxy-Role.
  • Copy the Role ARN: You will need this for the next step.

C. Create the Listener (API Gateway)

This creates the public HTTPS URL that AAP will talk to.

  1. Navigate to API Gateway.
  2. Create a REST API.
  3. Create Resource: Name it log (This makes your URL .../v1/log).
  4. Create Method: Select POST.
  5. Integration Type: AWS Service.
    • AWS Service: Firehose.
    • Action: PutRecord.
    • Execution Role: Paste the ARN of the APIGateway-Firehose-Proxy-Role created in Step 1B.
  6. Deploy API: Select a stage (e.g., v1).
  7. Copy the Invoke URL: (e.g., https://xyz123.execute-api.us-east-1.amazonaws.com/v1/log).

Step 2: Configure Sender (AAP)

Point Ansible Automation Platform to your new serverless collector.

  1. In AAP, navigate to Settings > Logging.
  2. Enable External Logging: Toggle to On.
  3. Logging Aggregator Type: Select Splunk (This format is JSON-compatible with AWS).
  4. Logging Aggregator: Enter the Invoke URL you copied from API Gateway (e.g., https://xyz123.../v1/log).
  5. Logging Aggregator Port: Leave blank or set to 443 (HTTPS default).
  6. Logging Aggregator Protocol: HTTPS.
  7. Click Save and Test. (The button should turn Green).

Step 3: Configure Data Analysis (Amazon Athena)

Unlike a VM that stores text files, S3 requires a query engine to read the data. We use Amazon Athena to turn your log bucket into a searchable database.

  1. Navigate to Amazon Athena.
  2. Open the Query Editor.
  3. Run the following SQL command to create a "Raw" table. This method is recommended to avoid JSON parsing errors with system logs.
CREATE EXTERNAL TABLE IF NOT EXISTS aap_raw_debug (
  raw_line string
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\0'
LINES TERMINATED BY '\n'
LOCATION 's3://aap-automation-logs-2026/';

Step 4: Validation (Visual Verification)

Logs will now flow from AAP ➔ API Gateway ➔ Firehose ➔ S3. To verify the pipeline is working, use Amazon Athena to confirm that a specific automation job completed successfully.

Validation Query

Run this SQL query in Athena. It specifically checks for the "Finalize Run" event, which confirms the controller finished processing a job, while filtering out system noise.

SELECT 
  -- 1. Get the Timestamp (Cast to string to prevent serialization errors)
  CAST(from_unixtime(CAST(json_extract_scalar(raw_line, '$.time') AS double)) AS VARCHAR) as Time,
  -- 2. Extract the Success Message
  json_extract_scalar(raw_line, '$.event.message') as Message,
  -- 3. Extract the Final Status (e.g., 'finalize_run')
  json_extract_scalar(raw_line, '$.event.lifecycle_data.state') as Status
FROM aap_raw_debug
WHERE raw_line LIKE '%finalize_run%'     -- Filter for the completion event
ORDER BY Time DESC
LIMIT 1;

Expected Result

You should see a single row confirming the latest job completion:
Visual Verification of AAP Logs in Athena:
 

Sample JSON Payload:

This is the raw data captured in S3 that generated the output above:

{"@timestamp":"2026-01-16T01:04:47.612Z","message":"projectupdate-611 waiting {\"type\": \"projectupdate\", \"task_id\": 611, \"state\": \"waiting\", \"work_unit_id\": null, \"task_name\": \"Demo Project\"}","host":"automation-controller-task-79d47497cf-q7xvj","level":"INFO","logger_name":"awx.analytics.job_lifecycle","lifecycle_data":{"type":"projectupdate","task_id":611,"state":"waiting","work_unit_id":null,"task_name":"Demo Project"},"organization_id":1,"cluster_host_id":"automation-controller-task-79d47497cf-q7xvj","tower_uuid":null}}

Supported Log Aggregators

While this guide uses AWS Native services, this architecture is compatible with forwarding to:

  • Splunk: Via Kinesis Firehose "Splunk" destination.
  • Datadog: Via Kinesis Firehose "Datadog" destination.
  • Elastic Stack (ELK): Via Kinesis Firehose "Elasticsearch" destination.
  • New Relic: Via Kinesis Firehose "New Relic" destination.

Reference Documentation

  • Amazon API Gateway as a Kinesis Proxy:
    https://docs.aws.amazon.com/apigateway/latest/developerguide/integrating-api-with-aws-services-kinesis.html
  • Querying JSON in Athena:
    https://docs.aws.amazon.com/athena/latest/ug/querying-JSON.html

Comments