Tracing Enabled

Risk Level: Medium (should be achieved)

Rule ID: Lambda-003

Ensure that active tracing is enabled for your Amazon Lambda functions in order to gain visibility into the execution and performance of the functions. With the tracing feature enabled, Amazon activates Lambda support for AWS X-Ray, a service that collects data about requests that your functions perform, which provides tools that you can use to view, filter, and gain insights into the collected data in order to identify issues and opportunities for optimization.

This rule can help you with the following compliance standards:

NIST4

For further details on compliance standards supported by Conformity, see here.

This rule can help you work with the AWS Well-Architected Framework.

This rule resolution is part of the Conformity Security & Compliance tool for AWS.

Operational
excellence

Sustainability

AWS X-Ray can provide tracing and monitoring capabilities for your Lambda functions. With active tracing mode enabled, you can save time and effort debugging and operating your functions as the X-Ray service support allows you to rapidly diagnose errors, identify bottlenecks, slowdowns and timeouts, by breaking down the latency for your Lambda functions.

Audit

To determine if active tracing is enabled for your Amazon Lambda functions, perform the following operations:

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to Amazon Lambda console at https://console.aws.amazon.com/lambda/.

03 In the navigation panel, under AWS Lambda, choose Functions.

04 Click on the name (link) of the function that you want to examine.

05 Select the Configuration tab and choose Monitoring and operations tools from the left menu.

06 In the Monitoring and operations tools section, check the Active tracing status. If the feature status is set to Not enabled, the tracing feature is disabled, therefore the AWS X-Ray support for the selected Amazon Lambda function is not enabled.

07 Repeat steps no. 4 – 6 for each Lambda function available within the current AWS region.

08 Change the AWS cloud region from the console navigation bar and repeat the Audit process for other regions.

Using AWS CLI

01 Run list-functions command (OSX/Linux/UNIX) to list the name of each Amazon Lambda function available in the selected AWS cloud region:

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899

aws lambda list-functions
	--region us-east-1
  --output table
	--query 'Functions[*].FunctionName'

02 The command output should return a table with the requested function name(s):

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899

---------------------
|   ListFunctions   |
+-------------------+
|   cc-sqs-poller   |
|   cc-s3-logging   |
|   s3-get-object   |
+-------------------+

03 Run get-function-configuration command (OSX/Linux/UNIX) using the name of the Amazon Lambda function that you want to examine as the identifier parameter and custom query filters to describe the tracing feature status (mode) available for the selected function:

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899

aws lambda get-function-configuration
  --region us-east-1
  --function-name cc-sqs-poller
  --query 'TracingConfig.Mode'

04 The command output should return the requested information (i.e. the tracing feature mode which can be either PassThrough or Active):

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899

"PassThrough"

If the status (mode) returned by the get-function-configuration command output is "PassThrough", as shown in the example above, the tracing feature is disabled, therefore the AWS X-Ray support for the selected Amazon Lambda function is not enabled.

05 Repeat step no. 3 and 4 for each Lambda function available in the selected AWS region.

06 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 – 5 to perform the Audit process for other regions.

Remediation / Resolution

To enable active tracing for your existing Amazon Lambda functions and make use of the AWS X-Ray support, perform the following operations:

Using AWS CloudFormation

01 CloudFormation template (JSON):

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899

{
  "AWSTemplateFormatVersion":"2010-09-09",
  "Description": "Enable Active (X-Ray) Tracing",
  "Resources":{
    "FunctionExecutionRole": {
        "Type": "AWS::IAM::Role",
        "Properties": {
          "RoleName": "LambdaExecutionRole",
          "AssumeRolePolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [{
              "Effect": "Allow",
              "Principal": {
                "Service": [ "lambda.amazonaws.com" ]
              },
              "Action": [ "sts:AssumeRole" ]
            }]
          },
          "Path": "/",
          "Policies": [{
            "PolicyName": "AWSLambdaBasicExecutionRole",
            "PolicyDocument": {
              "Version": "2012-10-17",
              "Statement": [{
                "Effect": "Allow",
                "Action": [
                  "logs:CreateLogGroup",
                  "logs:CreateLogStream",
                  "logs:PutLogEvents",
                  "xray:PutTraceSegments",
                  "xray:PutTelemetryRecords",
                  "xray:GetSamplingRules",
                  "xray:GetSamplingTargets",
                  "xray:GetSamplingStatisticSummaries"
                ],
                "Resource": "*"
              }]
            }
          }]
       }
    },
    "ConsumerFunction": {
        "Type": "AWS::Lambda::Function",
        "Properties": {
            "FunctionName": "cc-sqs-poller",
            "Handler": "index.handler",
            "Role": {
                "Fn::GetAtt": [
                    "FunctionExecutionRole",
                    "Arn"
                ]
            },
            "Code": {
                "S3Bucket": "cc-lambda-functions",
                "S3Key": "sqs-consumer.zip"
            },
            "Runtime": "nodejs12.x",
            "MemorySize" : 1024,
            "Timeout": 45,
            "TracingConfig": {
                "Mode": "Active"
            }
        }
    }
  }
}

02 CloudFormation template (YAML):

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899

AWSTemplateFormatVersion: '2010-09-09'
Description: Enable Active (X-Ray) Tracing
Resources:
  FunctionExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: LambdaExecutionRole
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - sts:AssumeRole
      Path: /
      Policies:
        - PolicyName: AWSLambdaBasicExecutionRole
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - logs:CreateLogGroup
                  - logs:CreateLogStream
                  - logs:PutLogEvents
                  - xray:PutTraceSegments
                  - xray:PutTelemetryRecords
                  - xray:GetSamplingRules
                  - xray:GetSamplingTargets
                  - xray:GetSamplingStatisticSummaries
                Resource: '*'
  ConsumerFunction:
    Type: AWS::Lambda::Function
    Properties:
      FunctionName: cc-sqs-poller
      Handler: index.handler
      Role: !GetAtt 'FunctionExecutionRole.Arn'
      Code:
        S3Bucket: cc-lambda-functions
        S3Key: sqs-consumer.zip
      Runtime: nodejs12.x
      MemorySize: 1024
      Timeout: 45
      TracingConfig:
        Mode: Active

Using Terraform (AWS Provider)

01 Terraform configuration file (.tf):

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.27"
    }
  }

  required_version = ">= 0.14.9"
}

provider "aws" {
  profile = "default"
  region  = "us-east-1"
}

resource "aws_iam_role" "function-execution-role" {
  name = "LambdaExecutionRole"
  path = "/"
  managed_policy_arns = [ "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole", "arn:aws:iam::aws:policy/AWSXrayWriteOnlyAccess" ]

  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Effect": "Allow"
    }
  ]
}
EOF
}

resource "aws_lambda_function" "lambda-function" {
  function_name    = "cc-sqs-poller"
  s3_bucket        = "cc-lambda-functions"
  s3_key           = "sqs-consumer.zip"
  role             = aws_iam_role.function-execution-role.arn
  handler          = "index.handler"
  runtime          = "nodejs12.x"
  memory_size      = 1024
  timeout          = 45

  # Enable Active (X-Ray) Tracing
  tracing_config {
    mode = "Active"
  }

}

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to Amazon Lambda console at https://console.aws.amazon.com/lambda/.

03 In the left navigation panel, under AWS Lambda, choose Functions.

04 Click on the name of the function that you want to reconfigure.

05 Select the Configuration tab and choose Monitoring and operations tools from the left menu.

06 In the Monitoring and operations tools section, choose Edit to change the monitoring configuration for the selected function.

07 In the AWS X-Ray section, toggle the Active tracing button to enable the tracing feature (i.e. activate AWS X-Ray support) for the selected Amazon Lambda function. When you enable the feature using the AWS Management Console, Amazon Lambda adds the required permissions (i.e. "xray:PutTraceSegments" and "xray:PutTelemetryRecords") to your function's execution role. Choose Save to apply the changes. Once the selected function is triggered, traces will begin to be generated and captured, allowing you to identify and address errors and exceptions, performance bottlenecks, and throttling.

08 Repeat steps no. 4 – 7 to enable active tracing for each Amazon Lambda function available within the current AWS region.

09 Change the AWS cloud region from the navigation bar and repeat the Remediation process for the other regions.

Using AWS CLI

01 Run get-function command (OSX/Linux/UNIX) to obtain the Amazon Resource Name (ARN) of the execution role associated with the Lambda function that you want to reconfigure:

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899

aws lambda get-function
  --region us-east-1
  --function-name cc-sqs-poller
  --query 'Configuration.Role'

02 The command output should return the requested role ARN:

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899

"arn:aws:iam::123456789012:role/service-role/cc-sqs-poller-role-abcdabcd"

03 Run attach-role-policy command (OSX/Linux/UNIX) to attach the "AWSXrayWriteOnlyAccess" managed IAM policy to the function's execution role, returned at the previous step. The "AWSXrayWriteOnlyAccess" policy gives AWS X-Ray service the permissions to upload trace data (the command does not produce an output):

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899

aws iam attach-role-policy
  --role-name cc-sqs-poller-role-abcdabcd
  --policy-arn "arn:aws:iam::aws:policy/AWSXrayWriteOnlyAccess"

04 Run update-function-configuration command (OSX/Linux/UNIX) using the name of the Amazon Lambda function that you want to reconfigure as the identifier parameter, to enable the tracing feature (i.e. activate AWS X-Ray support) for the selected function:

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899

aws lambda update-function-configuration
  --region us-east-1
  --function-name cc-sqs-poller
  --tracing-config '{"Mode":"Active"}'

05 The command output should return the metadata available for the reconfigured function:

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899

{
  "TracingConfig": {
      "Mode": "Active"
  },
  "FunctionName": "cc-sqs-poller",
  "CodeSize": 615,
  "MemorySize": 1024,
  "FunctionArn": "arn:aws:lambda:us-east-1:123456789012:function:cc-sqs-poller",
  "Version": "$LATEST",
  "Role": "arn:aws:iam::123456789012:role/service-role/LambdaS3Role",
  "Timeout": 45,
  "LastModified": "2021-08-30T10:00:00.000+0000",
  "Handler": "lambda_function.lambda_handler",
  "Runtime": "python3.7",
  "Description": ""
}

06 Repeat steps no. 1 – 5 to enable active tracing for each Amazon Lambda function available in the selected AWS region.

07 Change the AWS cloud region by updating the --region command parameter value and repeat steps no. 1 – 6 to perform the Remediation process for other regions.

References

AWS Command Line Interface (CLI) Documentation
lambda
list-functions
get-function
update-function-configuration
iam
attach-role-policy

CloudFormation Documentation
AWS Lambda resource type reference

Terraform Documentation
AWS Provider

Publication date Jun 12, 2017

Audit

Using AWS Console

Using AWS CLI

Remediation / Resolution

Using AWS CloudFormation

Using Terraform (AWS Provider)

Using AWS Console

Using AWS CLI

References

Related Lambda rules