- Knowledge Base
- Amazon Web Services
- AWS CloudTrail
- CloudTrail Delivery Failing
Ensure that the log files created by your Amazon CloudTrail trails are delivered without any failures to the designated recipients in order to keep CloudTrail log data for security and compliance audits.
This rule can help you with the following compliance standards:
- APRA
- MAS
- NIST4
For further details on compliance standards supported by Conformity, see here.
This rule can help you work with the AWS Well-Architected Framework.
This rule resolution is part of the Conformity Security & Compliance tool for AWS.
excellence
When your Amazon CloudTrail trails are not able to deliver log files to their recipients due to delivery errors or misconfigurations (usually involving the access policies that you have in place), the logging data recorded by these trails can't be saved and used for future security audits.
Audit
Case A: To identify Amazon CloudTrail trails that are not able to deliver log files to the designated S3 bucket(s), perform the following actions:
Using AWS Console
01 Sign in to the AWS Management Console.
02 Navigate to Amazon CloudTrail console at https://console.aws.amazon.com/cloudtrail/.
03 In the navigation panel, under CloudTrail, choose Trails.
04 Click on the name (link) of the Amazon CloudTrail trail that you want to examine.
05 In the General details section, verify the Last log file delivered attribute value (date/time). If the Last log file delivered value has not been updated recently (log files are generally published every 5 minutes), the selected Amazon CloudTrail trail failed to deliver the last log file to the designated S3 bucket.
06 Repeat steps no. 4 and 5 for each Amazon CloudTrail trail created within your AWS cloud account.
Using AWS CLI
01 Run list-trails command (OSX/Linux/UNIX) with custom query filters to list the names of all the Amazon CloudTrail trails created for your AWS cloud account:
aws cloudtrail list-trails --region us-east-1 --query 'Trails[*].Name'
02 The command output should return an array with the requested CloudTrail trail names:
[ "cc-main-cloud-trail", "cc-data-events-trail" ]
03 Run get-trail-status command (OSX/Linux/UNIX) using the name of the Amazon CloudTrail trail that you want to examine as the identifier parameter and custom query filters to expose any log file delivery errors available for the selected trail:
aws cloudtrail get-trail-status --region us-east-1 --name cc-main-cloud-trail --query 'LatestDeliveryError'
04 The command output should return null if the selected trail delivered successfully its last log file or an error code if the specified trail failed to deliver the log file:
"AccessDenied"
If the get-trail-status command responds with an error code instead of null, as shown in the output example above, the selected Amazon CloudTrail trail failed to deliver the last log file to the target S3 bucket.
05 Repeat steps no. 3 and 4 for each Amazon CloudTrail trail available in your AWS cloud account.
Case B: To determine if your Amazon CloudTrail trails are able to send SNS notifications for log file delivery, perform the following actions:
Using AWS Console
01 Sign in to the AWS Management Console.
02 Navigate to Amazon CloudTrail console at https://console.aws.amazon.com/cloudtrail/.
03 In the navigation panel, under CloudTrail, choose Trails.
04 Click on the name (link) of the Amazon CloudTrail trail that you want to examine.
05 In the General details section, verify the Last SNS notification attribute value. If the Last SNS notification attribute does not have a value, the selected Amazon CloudTrail trail failed to send SNS notifications for log file delivery.
06 Repeat steps no. 4 and 5 for each Amazon CloudTrail trail created within your AWS cloud account.
Using AWS CLI
01 Run list-trails command (OSX/Linux/UNIX) with custom query filters to list the names of all the Amazon CloudTrail trails created for your AWS cloud account:
aws cloudtrail list-trails --region us-east-1 --query 'Trails[*].Name'
02 The command output should return an array with the requested CloudTrail trail names:
[ "cc-main-cloud-trail", "cc-data-events-trail" ]
03 Run get-trail-status command (OSX/Linux/UNIX) using the name of the Amazon CloudTrail trail that you want to examine as the identifier parameter and custom query filters to expose any SNS notification errors triggered for the selected trail:
aws cloudtrail get-trail-status --region us-east-1 --name cc-main-cloud-trail --query 'LatestNotificationError'
04 The command output should return null if the selected trail is sending an SNS notification for each log file delivery or an error code if the trail failed to send SNS notifications:
"AuthorizationError"
If the get-trail-status command responds with an error code instead of null, as shown in the output example above, the selected Amazon CloudTrail trail failed to send SNS notifications for log file delivery.
05 Repeat steps no. 3 and 4 for each Amazon CloudTrail trail available in your AWS cloud account.
Remediation / Resolution
Case A: Usually, the Amazon CloudTrail trails fail to deliver log files when there is a problem with the target S3 bucket and will not occur for timeouts. To remediate this issue, create a new S3 bucket and associate your trail with the new bucket so that Amazon CloudTrail can send again log files to S3. To reconfigure your Amazon CloudTrail trails, perform the following actions:
Using AWS CloudFormation
01 CloudFormation template (JSON):
{ "AWSTemplateFormatVersion": "2010-09-09", "Description": "Replace the Target S3 Bucket with a New Bucket", "Parameters": { "TrailName": { "Type": "String" } }, "Resources": { "TargetS3Bucket": { "Type": "AWS::S3::Bucket", "Properties": { "BucketName": "cc-main-cloudtrail-bucket", "AccessControl": "Private", "PublicAccessBlockConfiguration": { "BlockPublicAcls": true, "IgnorePublicAcls": true, "BlockPublicPolicy": true, "RestrictPublicBuckets": true } } }, "TargetS3BucketPolicy": { "Type": "AWS::S3::BucketPolicy", "Properties": { "Bucket": "cc-main-cloudtrail-bucket", "PolicyDocument": { "Version": "2012-10-17", "Statement": [ { "Sid": "CloudTrailAclCheck", "Effect": "Allow", "Principal": { "Service": "cloudtrail.amazonaws.com" }, "Action": "s3:GetBucketAcl", "Resource": "arn:aws:s3:::cc-main-cloudtrail-bucket" }, { "Sid": "AWSCloudTrailWrite", "Effect": "Allow", "Principal": { "Service": "cloudtrail.amazonaws.com" }, "Action": "s3:PutObject", "Resource": "arn:aws:s3:::cc-main-cloudtrail-bucket/cc-trail-logs/AWSLogs/466594415815/*", "Condition": { "StringEquals": { "s3:x-amz-acl": "bucket-owner-full-control" } } } ] } } }, "Trail": { "Type": "AWS::CloudTrail::Trail", "Properties": { "TrailName": { "Ref": "TrailName" }, "S3BucketName": "cc-main-cloudtrail-bucket", "S3KeyPrefix": "cc-trail-logs", "IsLogging": true } } } }
02 CloudFormation template (YAML):
AWSTemplateFormatVersion: '2010-09-09' Description: Replace the Target S3 Bucket with a New Bucket Parameters: TrailName: Type: String Resources: TargetS3Bucket: Type: AWS::S3::Bucket Properties: BucketName: cc-main-cloudtrail-bucket AccessControl: Private PublicAccessBlockConfiguration: BlockPublicAcls: true IgnorePublicAcls: true BlockPublicPolicy: true RestrictPublicBuckets: true TargetS3BucketPolicy: Type: AWS::S3::BucketPolicy Properties: Bucket: cc-main-cloudtrail-bucket PolicyDocument: Version: '2012-10-17' Statement: - Sid: CloudTrailAclCheck Effect: Allow Principal: Service: cloudtrail.amazonaws.com Action: s3:GetBucketAcl Resource: arn:aws:s3:::cc-main-cloudtrail-bucket - Sid: AWSCloudTrailWrite Effect: Allow Principal: Service: cloudtrail.amazonaws.com Action: s3:PutObject Resource: arn:aws:s3:::cc-main-cloudtrail-bucket/cc-trail-logs/AWSLogs/466594415815/* Condition: StringEquals: s3:x-amz-acl: bucket-owner-full-control Trail: Type: AWS::CloudTrail::Trail Properties: TrailName: !Ref 'TrailName'
Using Terraform (AWS Provider)
01 Terraform configuration file (.tf):
terraform { required_providers { aws = { source = "hashicorp/aws" version = "~> 3.27" } } required_version = ">= 0.14.9" } provider "aws" { profile = "default" region = "us-east-1" } data "aws_caller_identity" "current-account" {} resource "aws_s3_bucket" "new-s3-bucket" { bucket = "cc-main-cloudtrail-bucket" force_destroy = true policy = <<POLICY { "Version": "2012-10-17", "Statement": [ { "Sid": "AWSCloudTrailAclCheck", "Effect": "Allow", "Principal": { "Service": "cloudtrail.amazonaws.com" }, "Action": "s3:GetBucketAcl", "Resource": "arn:aws:s3:::cc-main-cloudtrail-bucket" }, { "Sid": "AWSCloudTrailWrite", "Effect": "Allow", "Principal": { "Service": "cloudtrail.amazonaws.com" }, "Action": "s3:PutObject", "Resource": "arn:aws:s3:::cc-main-cloudtrail-bucket/cc-trail-logs/AWSLogs/${data.aws_caller_identity.current-account.account_id}/*", "Condition": { "StringEquals": { "s3:x-amz-acl": "bucket-owner-full-control" } } } ] } POLICY } resource "aws_cloudtrail" "cloudtrail-trail" { name = "cc-main-cloud-trail" # Replace the Target S3 Bucket with a New Bucket s3_bucket_name = aws_s3_bucket.new-s3-bucket.id s3_key_prefix = "cc-trail-logs" enable_logging = true }
Using AWS Console
01 Sign in to the AWS Management Console.
02 Navigate to Amazon CloudTrail console at https://console.aws.amazon.com/cloudtrail/.
03 In the navigation panel, under CloudTrail, choose Trails.
04 Click on the name (link) of the Amazon CloudTrail trail that you want to reconfigure.
05 In the General details section, choose Edit and perform the following operations:
- Select Create new S3 bucket under Storage location to create the new target bucket.
- Provide a unique name for the new S3 bucket in the Trail log bucket and folder box.
- Choose Save changes to apply the changes. Once the target bucket is created, Amazon CloudTrail starts to deliver log files to the new S3 bucket and the Last log file delivered attribute value is updated.
06 Repeat steps no. 4 and 5 for each Amazon CloudTrail trail that you want to reconfigure, available in your AWS cloud account.
Using AWS CLI
01 Run create-bucket command (OSX/Linux/UNIX) to create a new target S3 bucket for your Amazon CloudTrail trail:
aws s3api create-bucket --bucket cc-main-cloudtrail-bucket --region us-east-1 --acl private
02 The command output should return the name of the newly created S3 bucket:
{ "Location": "/cc-main-cloudtrail-bucket" }
03 Run put-public-access-block command (OSX/Linux/UNIX) to enable the S3 Public Access Block feature for the new Amazon S3 bucket (the command should not produce an output):
aws s3api put-public-access-block --region us-east-1 --bucket cc-main-cloudtrail-bucket --public-access-block-configuration BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true
04 Run update-trail command (OSX/Linux/UNIX) using the name of the Amazon CloudTrail trail that you want to reconfigure as the identifier parameter, to associate the new S3 bucket (i.e. target bucket) with the selected trail:
aws cloudtrail update-trail --region us-east-1 --name cc-main-cloud-trail --s3-bucket-name cc-main-cloudtrail-bucket
05 The command output should return the metadata available for the reconfigured trail:
{ "IncludeGlobalServiceEvents": true, "IsOrganizationTrail": false, "Name": "cc-main-cloud-trail", "TrailARN": "arn:aws:cloudtrail:us-east-1:123456789012:trail/cc-main-cloud-trail", "LogFileValidationEnabled": true, "IsMultiRegionTrail": true, "S3BucketName": "cc-main-cloudtrail-bucket" }
06 Repeat steps no. 1 – 5 for each Amazon CloudTrail trail that you want to reconfigure, created within your AWS cloud account.
Case B: When an Amazon CloudTrail trail fails to send SNS notifications for log file delivery, it's usually because there is a problem with the associated SNS topic. To resolve this issue, create a new SNS topic and associate your trail with the new topic so that Amazon CloudTrail can send delivery notifications again. To reconfigure your Amazon CloudTrail trails, perform the following actions:
Using AWS CloudFormation
01 CloudFormation template (JSON):
{ "AWSTemplateFormatVersion": "2010-09-09", "Description": "Replace the Associated SNS Topic with a New Topic", "Parameters": { "TrailName": { "Type": "String" }, "BucketName": { "Type": "String" }, "S3BucketKeyPrefix": { "Type": "String" } }, "Resources": { "Trail": { "Type": "AWS::CloudTrail::Trail", "Properties": { "TrailName": { "Ref": "TrailName" }, "S3BucketName": { "Ref": "BucketName" }, "S3KeyPrefix": { "Ref": "S3BucketKeyPrefix" }, "IsLogging": true, "SnsTopicName": "arn:aws:sns:us-east-1:123456789012:cc-cloud-trail-new-topic" } } } }
02 CloudFormation template (YAML):
AWSTemplateFormatVersion: '2010-09-09' Description: Replace the Associated SNS Topic with a New Topic Parameters: TrailName: Type: String BucketName: Type: String S3BucketKeyPrefix: Type: String Resources: Trail: Type: AWS::CloudTrail::Trail Properties: TrailName: !Ref 'TrailName' S3BucketName: !Ref 'BucketName' S3KeyPrefix: !Ref 'S3BucketKeyPrefix' IsLogging: true SnsTopicName: arn:aws:sns:us-east-1:123456789012:cc-cloud-trail-new-topic
Using Terraform (AWS Provider)
01 Terraform configuration file (.tf):
terraform { required_providers { aws = { source = "hashicorp/aws" version = "~> 3.27" } } required_version = ">= 0.14.9" } provider "aws" { profile = "default" region = "us-east-1" } data "aws_caller_identity" "current-account" {} resource "aws_s3_bucket" "trail-s3-bucket" { bucket = "cc-main-cloudtrail-bucket" force_destroy = true policy = <<POLICY { "Version": "2012-10-17", "Statement": [ { "Sid": "AWSCloudTrailAclCheck", "Effect": "Allow", "Principal": { "Service": "cloudtrail.amazonaws.com" }, "Action": "s3:GetBucketAcl", "Resource": "arn:aws:s3:::cc-main-cloudtrail-bucket" }, { "Sid": "AWSCloudTrailWrite", "Effect": "Allow", "Principal": { "Service": "cloudtrail.amazonaws.com" }, "Action": "s3:PutObject", "Resource": "arn:aws:s3:::cc-main-cloudtrail-bucket/cc-trail-logs/AWSLogs/${data.aws_caller_identity.current-account.account_id}/*", "Condition": { "StringEquals": { "s3:x-amz-acl": "bucket-owner-full-control" } } } ] } POLICY } resource "aws_cloudtrail" "cloudtrail-trail" { name = "cc-main-cloud-trail" s3_bucket_name = aws_s3_bucket.trail-s3-bucket.id s3_key_prefix = "cc-trail-logs" enable_logging = true # Replace the Associated SNS Topic with a New Topic sns_topic_name = "arn:aws:sns:us-east-1:123456789012:cc-cloud-trail-new-topic" }
Using AWS Console
01 Sign in to the AWS Management Console.
02 Navigate to Amazon CloudTrail console at https://console.aws.amazon.com/cloudtrail/.
03 In the navigation panel, under CloudTrail, choose Trails.
04 Click on the name (link) of the Amazon CloudTrail trail that you want to reconfigure.
05 In the General details section, choose Edit and perform the following operations:
- Select New under Create a new SNS topic to create a new Amazon SNS topic and associate it with the selected trail.
- Provide a unique name for the new topic in the SNS topic box.
- Choose Save changes to apply the changes. The new Amazon SNS topic will get permissions to receive notifications whenever a trail log file is delivered to the target S3 bucket.
06 Navigate to Amazon SNS console at https://console.aws.amazon.com/sns/.
07 In the navigation panel, under Amazon SNS, choose Topics.
08 Click on the name of the SNS topic created at step no. 5, select the Subscriptions tab, and choose Create subscription.
09 On the Create subscription setup page, select Email from the Protocol dropdown list, provide the email address where you want to receive trail log notifications in the Endpoint box, then choose Create subscription to apply the new subscription to the selected Amazon SNS topic.
10 Use your preferred email client to open the subscription message from the AWS Notifications, then click on the appropriate link to confirm your SNS subscription.
11 Repeat steps no. 4 – 10 for each Amazon CloudTrail trail that you want to reconfigure, available in your AWS cloud account.
Using AWS CLI
01 Run create-topic command (OSX/Linux/UNIX) to create a new SNS topic for your Amazon CloudTrail trail:
aws sns create-topic --region us-east-1 --name cc-cloud-trail-new-topic
02 The command output should return the Amazon Resource Name (ARN) of the newly created SNS topic:
{ "TopicArn": "arn:aws:sns:us-east-1:123456789012:cc-cloud-trail-new-topic" }
03 Run subscribe command (OSX/Linux/UNIX) to subscribe to the Amazon SNS topic created at the previous step using one or more email addresses as subscription endpoints:
aws sns subscribe --region us-east-1 --topic-arn arn:aws:sns:us-east-1:123456789012:cc-cloud-trail-new-topic --protocol email --notification-endpoint alert@cloudconformity.com --return-subscription-arn
04 The command output should return the ARN of the new SNS subscription:
{ "SubscriptionArn": "arn:aws:sns:us-east-1:123456789012:cc-cloud-trail-new-topic:abcdabcd-1234-abcd-1234-abcd1234abcd" }
05 Run confirm-subscription command (OSX/Linux/UNIX) to confirm the new SNS subscription by validating the token sent to the subscription endpoint (i.e. your email address) specified at the previous step (the command should not produce an output):
aws sns confirm-subscription --region us-east-1 --topic-arn arn:aws:sns:us-east-1:123456789012:cc-cloud-trail-new-topic --token 5014392f37fb687f5d51e6e241d7700ae02f7124d8268910b858cb4db727ceeb2474bb937929d3bdd7ce5d0cce19325d036bca58d3c217426bcafa9c501a2cac5646456gf1dd3797627467553dc438a8c974119496fc3eff026eaa5d15578ded6f9a5c43aec62d83ef5f49109da730881
06 Run update-trail command (OSX/Linux/UNIX) using the name of the Amazon CloudTrail trail that you want to reconfigure as the identifier parameter, to associate the selected trail with the newly created SNS topic:
aws cloudtrail update-trail --region us-east-1 --name cc-main-cloud-trail --sns-topic-name cc-cloud-trail-new-topic
07 The command output should return the metadata available for the reconfigured trail:
{ "IncludeGlobalServiceEvents": true, "IsOrganizationTrail": false, "Name": "cc-main-cloud-trail", "TrailARN": "arn:aws:cloudtrail:us-east-1:123456789012:trail/cc-main-cloud-trail", "LogFileValidationEnabled": false, "SnsTopicARN": "arn:aws:sns:us-east-1:123456789012:cc-cloud-trail-new-topic", "IsMultiRegionTrail": true, "S3BucketName": "aws-cloudtrail-log-bucket", "SnsTopicName": "cc-cloud-trail-new-topic" }
08 Repeat steps no. 1 – 7 for each Amazon CloudTrail trail that you want to reconfigure, created within your AWS cloud account.
References
- AWS Documentation
- AWS CloudTrail FAQs
- GetTrailStatus
- Error Responses
- Getting and Viewing Your CloudTrail Log Files
- Configuring Amazon SNS Notifications for CloudTrail
- What Is AWS CloudTrail?
- Updating a Trail
- AWS Command Line Interface (CLI) Documentation
- cloudtrail
- list-trails
- get-trail-status
- update-trail
- sns
- create-topic
- subscribe
- confirm-subscription
- s3api
- create-bucket
- put-public-access-block
- CloudFormation Documentation
- AWS CloudTrail resource type reference
- Terraform Documentation
- AWS Provider
Related CloudTrail rules
Unlock the Remediation Steps
Free 30-day Trial
Automatically audit your configurations with Conformity
and gain access to our cloud security platform.
You are auditing:
CloudTrail Delivery Failing
Risk Level: Medium