RDS Free Storage Space

Risk Level: High (not acceptable risk)

Rule ID: RDS-022

Identify any Amazon RDS database instances that appear to run low on disk space and scale them up to alleviate any problems triggered by insufficient disk space and improve their I/O performance. The default threshold value set for the amount of free storage space is 10% as any value below this could have a serious impact on your database stability and performance. For example, if the free storage space becomes dangerously low, basic operations like connecting to the database will not be possible anymore.
The AWS CloudWatch metric utilized to detect RDS instances with low free storage space is:
FreeStorageSpace - the amount of available storage space. (Units: Bytes).

This rule can help you work with the AWS Well-Architected Framework.

This rule resolution is part of the Conformity Security & Compliance tool for AWS.

Sustainability

Performance
efficiency

Low disk space will often lead to instability and slowdowns. Detecting RDS database instances that run low on disk space is crucial when these instances are used in production by latency sensitive applications as this can help you take immediate actions and expand the storage space in order to maintain an optimal response time.

Note 1: You can change the default threshold value (10%) for this rule on Cloud Conformity console and set your own value for the amount of available storage space to configure the storage limits for your RDS instances.
Note 2: Scaling up the storage space is applicable only to the following RDS instance types: MySQL, MariaDB, PostgreSQL and Oracle DB.

Audit

To identify any existing RDS instances that run low on disk space, available in your AWS account, perform the following:

Using AWS Console

01 Log in to the AWS Management Console.

02 Navigate to RDS dashboard at https://console.aws.amazon.com/rds/.

03 In the left navigation panel, under RDS Dashboard section, choose Instances.

04 Select the RDS database instance that you want to examine.

05 Click Instance Actions button from the dashboard top menu and select See Details.

06 On the Details tab, in the Instance and IOPS section, check the Storage attribute value to get the amount of storage allocated for the selected database instance, in gigabytes.

07 Now go back to the RDS dashboard and select again the database instance that you want to examine.

08 Click on Show Monitoring button from the dashboard top menu and select Show Multi-Graph View to expand the AWS CloudWatch monitoring panel.

09 On the monitoring panel displayed for the selected instance, click on the Free Storage Space usage graph thumbnail to open the RDS instance free storage space details box. Inside the Free Storage Space (MB) dialog box, set the following parameters:

From the Statistic dropdown list, select Maximum.
From the Time Range list, select Last 24 Hours.
From the Period dropdown list, select 1 Hour.

Once the monitoring data is loaded, verify the free storage space currently available (megabytes) for the selected database instance.

10 Based on instance storage information returned at step no. 6 and 9, if the amount of free storage space is less than or equal to 10%, the selected AWS RDS database instance has insufficient disk space to perform optimally, therefore expanding the instance storage space is highly recommended.

11 Repeat steps no. 4 – 10 to determine the amount of free storage space for other RDS database instances provisioned within the current region.

12 Change the AWS region from the navigation bar and repeat the audit process for other regions.

Using AWS CLI

01 Run describe-instances command (OSX/Linux/UNIX) using custom query filters to list the IDs of all the RDS database instances available in the selected region:

aws rds describe-db-instances
	--region us-east-1
	--output table
	--query 'DBInstances[*].DBInstanceIdentifier'

02 The command output should return a table with the requested IDs:

-------------------------
|  DescribeDBInstances  |
+-----------------------+
|  cc-mysql-prod-db     |
|  cc-mysql-webdev-db   |
+-----------------------+

03 Run again describe-instances command (OSX/Linux/UNIX) using the ID (identifier) of the instance that you want to examine as identifier to get the instance allocated storage size specified in gigabytes:

aws rds describe-db-instances
	--region us-east-1
	--db-instance-identifier cc-mysql-prod-db
	--query 'DBInstances[*].AllocatedStorage'

04 The command output should return the storage size allocated for the selected instance (gigabytes)

[
    40
]

05 Run get-metric-statistics command (OSX/Linux/UNIX) to get the statistics recorded by AWS CloudWatch for the FreeStorageSpace metric representing the amount of available storage space (bytes) for the selected RDS instance. The following command example returns the amount of free storage space available for an RDS database instance identified by the ID cc-mysql-prod-db, data captured during a 24 hours time frame, using 1 hour time range as the granularity of the returned datapoints:

aws cloudwatch get-metric-statistics
	--region us-east-1
	--metric-name FreeStorageSpace
	--start-time 2017-09-10T20:05:00
	--end-time 2017-09-11T20:05:00
	--period 3600
	--namespace AWS/RDS
	--statistics Maximum
	--dimensions="Name=DBInstanceIdentifier,Value=cc-mysql-prod-db"

06 The command output should return the available storage space details requested:

{
    "Datapoints": [
        {
            "Timestamp": "2017-09-10T20:05:00Z",
            "Maximum": 2196401152.0,
            "Unit": "Bytes"
        },
        {
            "Timestamp": "2017-09-10T21:05:00Z",
            "Maximum": 2196433920.0,
            "Unit": "Bytes"
        },
        {
            "Timestamp": "2017-09-10T22:05:00Z",
            "Maximum": 2196401152.0,
            "Unit": "Bytes"
        },


        ...


        {
            "Timestamp": "2017-09-11T18:05:00Z",
            "Maximum": 2196405248.0,
            "Unit": "Bytes"
        },
        {
            "Timestamp": "2017-09-11T19:05:00Z",
            "Maximum": 2196405248.0,
            "Unit": "Bytes"
        },
        {
            "Timestamp": "2017-09-11T20:05:00Z",
            "Maximum": 2196433920.0,
            "Unit": "Bytes"
        }
    ],
    "Label": "FreeStorageSpace"
}

07 Based on instance storage information returned at step no. 4 and 6, if the amount of free storage space is less than or equal to 10%, the selected AWS RDS database instance has insufficient disk space to perform optimally.

08 Repeat steps no. 3 – 7 to determine the amount of free storage space for other RDS database instances created within the current region.

09 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 - 8 to perform the entire audit process for other regions.

Remediation / Resolution

To scale up (expand) the storage space for any RDS database instances that run low on disk space, perform the following actions:

Note: During the scaling process, the RDS database instance is available for use (no downtime), but its performance may be degraded.

Using AWS Console

01 Log in to the AWS Management Console.

02 Navigate to RDS dashboard at https://console.aws.amazon.com/rds/.

03 In the navigation panel, under RDS Dashboard section, choose Instances.

04 Select the RDS database instance that you want to reconfigure (see Audit section part I to identify the right resource).

05 Click the Instance Actions button from the dashboard top menu and select Modify.

06 On the Modify DB Instance: <instance identifier> page, under Instance Specifications section, increase the existing storage space (in gigabytes) by entering a new value inside the Allocated Storage box.

07 At the bottom of the page, check Apply Immediately to apply the allocated storage changes immediately.

08 Click Continue to access the database instance review page.

09 Review the storage changes made for the selected instance, then click Modify DB Instance. During the resize process the instance status should change from available to modifying and back to available (it may take few minutes depending on the database instance size and configuration). Once the scaling process is complete, the RDS instance storage size should be updated on the details panel.

10 Repeat steps no. 4 – 9 to increase the storage space for other RDS database instances that may run low on disk space, provisioned within the current region.

11 Change the AWS region from the navigation bar and repeat the remediation/resolution process for other regions.

Using AWS CLI

01 Run modify-db-instance command (OSX/Linux/UNIX) to expand the allocated storage space for the selected RDS database instance (see Audit section part II to identify the right RDS resource). The valid values (in gigabytes) for allocated storage space for different database engines can be found at this URL. The following command example make use of --apply-immediately parameter to apply the storage capacity change asynchronously. If you skip adding the --apply-immediately parameter to the command request, AWS RDS service will apply the storage change during the next maintenance window:

aws rds modify-db-instance
	--region us-east-1
	--db-instance-identifier cc-mysql-prod-db
	--allocated-storage 100
	--apply-immediately

02 The command output should return the new configuration metadata for the selected RDS database instance:

{
    "DBInstances": [
        {
            "PubliclyAccessible": true,
            "MonitoringInterval": 0,
            "LicenseModel": "general-public-license",
            "InstanceCreateTime": "2017-04-07T12:04:45.232Z",
            "CopyTagsToSnapshot": false,
            "Engine": "mysql",
            "MultiAZ": false,
            "AutoMinorVersionUpgrade": true,
            "ReadReplicaDBInstanceIdentifiers": [],
            "AllocatedStorage": 100,

            ...

            "DBInstanceStatus": "available",
            "IAMDatabaseAuthenticationEnabled": false,
            "EngineVersion": "5.6.27",
            "AvailabilityZone": "us-east-1",
            "DomainMemberships": [],
            "StorageType": "gp2",
            "CACertificateIdentifier": "rds-ca-2015",
            "StorageEncrypted": false,
            "DBInstanceClass": "db.m3.medium",
            "DBInstanceIdentifier": "cc-mysql-prod-db"
        }
    ]
}

03 Repeat step no. 1 and 2 to increase the storage space for other RDS database instances that may run low on disk space, available in the current region.

04 Change the AWS region by updating the --region command parameter value and repeat the entire process for other regions.

References

Publication date Jan 9, 2018

Audit

Using AWS Console

Using AWS CLI

Remediation / Resolution

Using AWS Console

Using AWS CLI

References

Related RDS rules