Kinesis Stream Shard Level Metrics

01 Sign in to the AWS Management Console.

02 Navigate to Kinesis dashboard at https://console.aws.amazon.com/kinesis/.

03 In the navigation panel, under Amazon Kinesis, choose Streams.

04 Select the Kinesis stream that you want to examine, click the Actions dropdown button and select Details to access the stream configuration details.

05 Choose the Details tab from the top panel and verify the Shard level metrics feature status available within the Shard level metrics section. If the status is set to "No shard level metrics enabled", the selected AWS Kinesis streams does not have any shard-level metrics currently enabled.

06 Repeat step no. 4 and 5 for each Amazon Kinesis stream available in the current AWS region.

07 Change the AWS region from the navigation bar to repeat the audit process for other regions.

01 Run list-streams command (OSX/Linux/UNIX) to list the names of all Kinesis streams available within the selected AWS region - US East (N. Virginia):

aws kinesis list-streams
	--region us-east-1
	--query 'StreamNames'

02 The command output should return the requested Kinesis stream names:

[
    "iot-kinesis-stream",
    "customers-data-stream"
]

03 Run describe-stream command (OSX/Linux/UNIX) using the stream name returned at the previous step as identifier and custom query filters to return the list of the shard-level metrics enabled for the selected Kinesis stream:

aws kinesis describe-stream
	--region us-east-1
	--stream-name iot-kinesis-stream
	--query 'StreamDescription.EnhancedMonitoring[*].ShardLevelMetrics[]'

04 The describe-stream command output should return an array that contains the name(s) of the shard-level metric(s) enabled for the Kinesis stream, the value "ALL" if the stream enhances every metric available and an empty array if there are no shard-level metrics activated. If the output returned is an empty array, i.e. [ ], the selected AWS Kinesis streams does not have any shard-level metrics enabled.

05 Repeat step no. 3 and 4 for each AWS Kinesis stream provisioned in the current AWS region.

06 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 - 5 to perform the entire process for other regions.

01 Sign in to the AWS Management Console.

02 Navigate to Kinesis dashboard at https://console.aws.amazon.com/kinesis/.

03 In the navigation panel, under Amazon Kinesis, choose Streams.

04 Select the Kinesis stream that you want to examine, click the Actions dropdown button and select Details to open the stream configuration page.

05 Select the Details tab from the top panel, locate the Shard level metrics section and click the Edit button next to it:

06 Check "All shard level metrics" checkbox to enable every shard-level metric available for the selected stream or select individual metrics based on your requirements. Full description for each shard-level metric is available at this URL.

07 Click Save to apply the configuration changes. Once the request is made, a pop-up message will be displayed: "Updating shard level metrics for the stream. This will take up to 70 seconds.". The selected stream transitions through a "pending" state. Once the Kinesis stream returns to the "active" state, the shard-level metrics that you selected become active.

08 Repeat steps no. 4 – 7 to enable enhanced monitoring for other Amazon Kinesis streams available in the current AWS region.

09 Change the AWS region from the navigation bar and repeat the process for other regions.

01 Run enable-enhanced-monitoring command (OSX/Linux/UNIX) using the name of the stream that you want to reconfigure (see Audit section part II to identify the right Kinesis resource) to enable enhanced monitoring for the selected Amazon Kinesis stream using valid shard-level metrics. The following command example enables all shard-level metrics made available by Amazon Kinesis for a stream named "iot-kinesis-stream", provisioned within the US East (N. Virginia) region:

aws kinesis enable-enhanced-monitoring
	--region us-east-1
	--stream-name iot-kinesis-stream
	--shard-level-metrics ALL

02 The command output should return the enhanced monitoring metadata which contains the name of each shard-level metric enabled for the selected AWS Kinesis stream:

{
    "StreamName": "iot-kinesis-stream",
    "CurrentShardLevelMetrics": [],
    "DesiredShardLevelMetrics": [
        "IncomingBytes",
        "OutgoingRecords",
        "IteratorAgeMilliseconds",
        "IncomingRecords",
        "ReadProvisionedThroughputExceeded",
        "WriteProvisionedThroughputExceeded",
        "OutgoingBytes"
    ]
}

03 Repeat step no. 1 and 2 to enable enhanced stream monitoring using shard-level metrics for other Amazon Kinesis streams created in the current AWS region.

04 Change the AWS region by updating the --region command parameter value and repeat the entire process for other regions.