Enable Diagnostic Logs for OpenAI Service Instances

Risk Level: Medium (should be achieved)

Rule ID: AIServices-003

To collect detailed information on resource operations, ensure that Diagnostic Logs are enabled for your Microsoft Azure OpenAI service instances. Diagnostic Logs provide detailed insights into operations, helps identify and resolve issues quickly, ensures compliance with governance policies, and supports auditing and analysis of resource usage and performance.

This rule resolution is part of the Conformity Security & Compliance tool for Azure.

Security

Reliability

Operational
excellence

Cost
optimisation

Performance
efficiency

By default, Diagnostic Logs are not enabled for your Azure OpenAI service instances. Without Diagnostic Logs, the visibility into your Azure data plane is greatly reduced. This diminishes your organization's ability to detect potential attacks, unauthorized requests, or other malicious activity. For example, without Diagnostic Logs, it would be difficult to tell which entities had accessed a breached data store. In addition, alerts for failed attempts to access APIs for Azure database services are only possible when diagnostic logging is enabled. Once collected, Diagnostic Logs should be sent to a storage account and a Log Analytics Workspace or an equivalent third-party system. The log files should be kept in readily accessible storage for at least one year, and then moved to inexpensive cold storage for a longer duration (for security and compliance auditing).

Audit

To determine if Diagnostic Logs are enabled for your Azure OpenAI service instances, perform the following actions:

Using Azure Console

01 Sign in to the Azure Management Console.

02 Navigate to Azure OpenAI blade at https://portal.azure.com/#view/Microsoft_Azure_ProjectOxford/CognitiveServicesHub/~/OpenAI.

03 Select the Azure subscription that you want to examine from the Subscription equals all filter box, and choose Apply.

04 Click on the name (link) of the Azure OpenAI service instance that you want to examine. An OpenAI service instance has Kind set to OpenAI.

05 In the resource navigation panel, under Monitoring, select Diagnostic settings.

06 In the Diagnostic settings section, check for any diagnostic settings created for the selected OpenAI instance. If there are no diagnostic settings available, the Diagnostic Logs monitoring feature is not enabled for the selected instance. If one or more diagnostic settings were created for your instance, choose the diagnostic setting that you want to examine, and select Edit settings. Check the Categories list under Logs to determine the logging configuration available for your OpenAI instance. If one or more log categories are not selected, Diagnostic Logs are not enabled for the selected Azure OpenAI service instance.

07 Repeat steps no. 4 – 6 for each Azure OpenAI instance available within the selected subscription.

08 Repeat steps no. 3 – 7 for each subscription created in your Microsoft Azure cloud account.

Using Azure CLI

01 Run cognitiveservices account list command (Windows/macOS/Linux) with custom output filters to list the identifier (ID) of each Azure OpenAI service instance available in the current subscription:

az cognitiveservices account list
  --query '[?(kind==`OpenAI`)].id'

02 The command output should return the requested OpenAI service instance identifiers:

[
	"/subscriptions/abcd1234-abcd-1234-abcd-1234abcd1234/resourceGroups/cloud-shell-storage-westeurope/providers/Microsoft.CognitiveServices/accounts/tm-openai-project5-instance",
	"/subscriptions/abcd1234-abcd-1234-abcd-1234abcd1234/resourceGroups/cloud-shell-storage-westeurope/providers/Microsoft.CognitiveServices/accounts/tm-openai-workspace-server"
]

03 Run monitor diagnostic-settings list command (Windows/macOS/Linux) with the ID of the Azure OpenAI service instance that you want to examine as the identifier parameter and custom output filters to describe the name of each diagnostics setting configured for the selected OpenAI instance:

az monitor diagnostic-settings list
  --resource "/subscriptions/abcd1234-abcd-1234-abcd-1234abcd1234/resourceGroups/cloud-shell-storage-westeurope/providers/Microsoft.CognitiveServices/accounts/tm-openai-project5-instance"
  --query '[].name'

04 The command output should return the requested diagnostics setting identifiers (names). If the command output returns an empty array, i.e. [], there are no diagnostic settings configured for the selected Azure OpenAI service instance, therefore, the Audit process ends here:

[
	"tm-instance-diagnostic-logs",
	"tm-custom-diagnostic-logs"
]

05 Run monitor diagnostic-settings show command (Windows/macOS/Linux) with the name of the diagnostic setting that you want to examine as the identifier parameter, to describe the log categories supported by Azure OpenAI, configured for the selected diagnostic setting:

az monitor diagnostic-settings show
  --name "tm-instance-diagnostic-logs"
  --resource "/subscriptions/abcd1234-abcd-1234-abcd-1234abcd1234/resourceGroups/cloud-shell-storage-westeurope/providers/Microsoft.CognitiveServices/accounts/tm-openai-project5-instance"
  --query 'logs'

06 The command output should return the supported log categories and their status:

[
	{
		"categoryGroup": "allLogs",
		"enabled": false,
		"retentionPolicy": {
		  "days": 0,
		  "enabled": false
		}
	},
	{
		"categoryGroup": "audit",
		"enabled": false,
		"retentionPolicy": {
		  "days": 0,
		  "enabled": false
		}
	}
]

07 Repeat steps no. 5 and 6 for each diagnostic setting created for the selected instance. If none of the diagnostic settings are properly configured, the Diagnostic Logs monitoring feature is not enabled for the selected Azure OpenAI service instance.

08 Repeat steps no. 3 - 7 for each Azure OpenAI instance available in the selected Azure subscription.

09 Repeat steps no. 1 – 8 for each subscription created in your Microsoft Azure cloud account.

Remediation / Resolution

To enable and configure Diagnostic Logs for your Microsoft Azure OpenAI service instance, perform the following operations:

Using Azure Console

01 Sign in to the Azure Management Console.

02 Navigate to Azure OpenAI blade at https://portal.azure.com/#view/Microsoft_Azure_ProjectOxford/CognitiveServicesHub/~/OpenAI.

03 Select the Azure subscription that you want to access from the Subscription equals all filter box, and choose Apply.

04 Click on the name (link) of the Azure OpenAI service instance that you want to configure.

05 In the resource navigation panel, under Monitoring, select Diagnostic settings.

06 In the Diagnostic settings section, choose + Add diagnostic setting to create a new diagnostic setting resource. A diagnostic setting specifies a list of categories of platform logs and/or metrics that you want to collect from an Azure cloud resource, and one or more destinations that you would stream them to.

07 On the Diagnostic setting setup page, perform the following actions:

Provide a unique name for your new diagnostic setting in the Diagnostic setting name box.
In the Logs section, check the allLogs setting checkbox to select all the log categories supported by Azure OpenAI.
(Optional) In the Metrics section, check the AllMetrics setting checkbox if you want to route the resource's platform metrics to the selected log destinations. By default, platform metrics are sent automatically to Azure Monitor Metrics without any additional configuration required.
In the Destination details section, perform the following operations:
1. Select Send to Log Analytics workspace and choose a workspace from the Log Analytics workspace dropdown list to send the diagnostic logs.
2. Select Archive to a storage account and choose a storage account from the Storage account dropdown list to archive the collected logs for at least one year (recommended). The storage account needs to be in the same region as the resource being monitored if the resource is regional.
3. (Optional) Select Stream to an event hub to stream the collected logs to an Event Hub.
4. (Optional) Select Send to partner solution to deliver the log files to a supported third-party system.
Choose Save to apply the configuration changes.

08 Repeat steps no. 4 – 7 for each Azure OpenAI service instance available within the selected subscription.

09 Repeat steps no. 3 – 8 for each subscription created in your Microsoft Azure cloud account.

Using Azure CLI

01 Run monitor diagnostic-settings create command (Windows/macOS/Linux) to create a new diagnostic setting for the specified Azure OpenAI service instance in order to enable the Diagnostic Logs monitoring feature. As an example, the following command request creates a diagnostic setting named "tm-project5-diagnostic-logs" for an Azure OpenAI instance, that sends the supported logs and metrics to a Log Analytics workspace identified by the ID "abcd1234abcd1234abcd1234" and archive the files to an Azure Storage account identified by "abcd1234abcd1234abcd1234". Platform metrics are sent automatically to Azure Monitor Metrics by default and without any configuration. If you need to route the resource's platform metrics to the selected log destinations, include the --metrics parameter in the command request, as shown in the example below:

az monitor diagnostic-settings create
  --name "tm-project5-diagnostic-logs"
  --resource "/subscriptions/abcd1234-abcd-1234-abcd-1234abcd1234/resourceGroups/cloud-shell-storage-westeurope/providers/Microsoft.CognitiveServices/accounts/tm-openai-project5-instance"
  --workspace "abcd1234abcd1234abcd1234"
  --storage-account "1234abcd1234abcd1234"
  --logs '[
			{
			"categoryGroup": "allLogs",
			"enabled": true,
			"retentionPolicy": {
				"days": 0,
				"enabled": false
			}
			},
			{
			"categoryGroup": "audit",
			"enabled": false,
			"retentionPolicy": {
				"days": 0,
				"enabled": false
			}
			}
		]'
		--metrics '[
		{
			"category": "AllMetrics",
			"enabled": true,
			"retentionPolicy": {
				"enabled": false,
				"days": 0
			}
		}
	]'

02 The command output should return the metadata available for the new diagnostic setting:

{
	"id": "/subscriptions/abcd1234-abcd-1234-abcd-1234abcd1234/resourcegroups/cloud-shell-storage-westeurope/providers/microsoft.cognitiveservices/accounts/tm-openai-project5-instance/providers/microsoft.insights/diagnosticSettings/tm-project5-diagnostic-logs",
	"logs": [
		{
			"categoryGroup": "allLogs",
			"enabled": true,
			"retentionPolicy": {
			"days": 0,
			"enabled": false
			}
		},
		{
			"categoryGroup": "audit",
			"enabled": false,
			"retentionPolicy": {
			"days": 0,
			"enabled": false
			}
		}
	],
	"metrics": [
		{
			"category": "AllMetrics",
			"enabled": true,
			"retentionPolicy": {
				"days": 0,
				"enabled": true
			},
			"timeGrain": "PT1M"
		}
	],
	"name": "tm-project5-diagnostic-logs",
	"resourceGroup": "cloud-shell-storage-westeurope",
	"storageAccountId": "/subscriptions/abcdabcd-1234-abcd-1234-abcd1234abcd/resourceGroups/cloud-shell-storage-westeurope/providers/microsoft.Storage/storageAccounts/1234abcd1234abcd1234",
	"type": "Microsoft.Insights/diagnosticSettings",
	"workspaceId": "/subscriptions/abcdabcd-1234-abcd-1234-abcd1234abcd/resourceGroups/cloud-shell-storage-westeurope/providers/microsoft.OperationalInsights/workspaces/abcd1234abcd1234abcd1234"
}

03 Repeat steps no. 1 and 2 for each Azure OpenAI service instance provisioned in the selected subscription.

04 Repeat steps no. 1 – 3 for each subscription created within your Microsoft Azure cloud account.

References

Azure Command Line Interface (CLI) Documentation
az cognitiveservices account list
az monitor diagnostic-settings list
az monitor diagnostic-settings show
az monitor diagnostic-settings create

Publication date Jun 21, 2024

Audit

Using Azure Console

Using Azure CLI

Remediation / Resolution

Using Azure Console

Using Azure CLI

References

Related AIServices rules