Use the Conformity Knowledge Base AI to help improve your Cloud Posture

Enable Managed Virtual Network Isolation with Internet Outbound Access

Trend Cloud One™ – Conformity is a continuous assurance tool that provides peace of mind for your cloud infrastructure, delivering over 1000 automated best practice checks.

Risk Level: High (not acceptable risk)
Rule ID: MachineLearning-005

Ensure that managed virtual network (managed VNet) isolation with Internet outbound is enabled for your Azure Machine Learning (ML) workspaces. This restricts public inbound traffic while allowing outbound traffic to the Internet from the provided managed VNet.

This rule resolution is part of the Conformity Security & Compliance tool for Azure.

Security

Enabling managed virtual network isolation with Internet outbound for Azure Machine Learning workspaces enhances security by restricting access to the workspace, ensuring that only authorized resources within the managed VNet can communicate with it while still allowing essential outbound Internet access for updates and necessary operations such as download Python packages or pretrained models. This isolation minimizes the risk of unauthorized access and data breaches.


Audit

To determine if managed VNet isolation with Internet outbound access is enabled for your Azure ML workspaces, perform the following actions:

Using Azure Console

01 Sign in to the Azure Management Console.

02 Navigate to Azure Machine Learning workspaces blade at https://portal.azure.com/#browse/Microsoft.MachineLearningServices%2Fworkspaces.

03 Select the Azure subscription that you want to examine from the Subscription filter box.

04 Click on the name (link) of the Azure Machine Learning workspace that you want to examine.

05 In the resource navigation panel, under Settings, choose Networking to access the networking settings available for the selected workspace.

06 On the Networking page, perform the following checks:

  1. Select the Public access tab and check the Public network access configuration setting. If Public network access is not set to Disabled, the workspace is accessible via public endpoints, therefore, managed virtual network (managed VNet) isolation is not enabled for the selected workspace.
  2. Select the Workspace managed outbound access tab and check the outbound mode configured for your workspace. If Allow Internet Outbound is not selected for the outbound mode, managed virtual network isolation with Internet outbound access is not enabled for the selected Azure Machine Learning (ML) workspace.

07 Repeat steps no. 4 – 6 for each Azure Machine Learning workspace available within the selected subscription.

08 Repeat steps no. 3 – 7 for each subscription created in your Microsoft Azure cloud account.

Using Azure CLI

01 Run ml workspace list command (Windows/macOS/Linux) with custom query filters to list the name and the associated resource group for each Azure Machine Learning workspace available in the current subscription:

az ml workspace list
  --output table
  --query '[*].{name:name, resourceGroup:resourceGroup}'

02 The command output should return the requested Azure Machine Learning workspace identifiers:

Name                           ResourceGroup
------------------------       ------------------------------
tm-ml-project5-workspace       cloud-shell-storage-westeurope
tm-ml-production-workspace     cloud-shell-storage-westeurope

03 Run ml workspace show command (Windows/macOS/Linux) with the name of the Azure Machine Learning workspace that you want to examine as the identifier parameter and custom output filters to describe the public network access status and the outbound mode configured for the selected workspace:

az ml workspace show
  --name tm-ml-project5-workspace
  --resource-group cloud-shell-storage-westeurope
  --query '{"public_access":public_network_access,"outbound_mode":managed_network.isolation_mode}'

04 The command output should return the requested networking information:

{
	"public_access": "Enabled",
	"outbound_mode": "disabled"
}

If the ml workspace show command output returns "Enabled" for "public_access", the workspace is accessible via public endpoints, therefore, managed virtual network (managed VNet) isolation is not enabled for the workspace. If the command output returns "disabled" for "outbound_mode", as shown in the example above, managed virtual network isolation with Internet outbound access is not enabled for the selected Azure Machine Learning (ML) workspace.

05 Repeat step no. 3 and 4 for each Azure Machine Learning workspace available in the selected Azure subscription.

06 Repeat steps no. 1 – 5 for each subscription created in your Microsoft Azure cloud account.

Remediation / Resolution

To enable managed virtual network isolation with Internet outbound access for your Azure Machine Learning (ML) workspaces, you must re-create your workspaces with the necessary network configuration, by performing the following operations:

Using Azure Console

01 Sign in to the Azure Management Console.

02 Navigate to Azure Machine Learning workspaces blade at https://portal.azure.com/#browse/Microsoft.MachineLearningServices%2Fworkspaces.

03 Select the Azure subscription that you want to access from the Subscription equals all filter box, and choose Apply.

04 Choose Create, select New workspace, and perform the following actions to create your new Azure Machine Learning workspace:

  1. For Basics, provide the following information:
    1. For Subscription, choose your Azure subscription.
    2. For Resource group, select the correct resource group.
    3. Provide a unique name for the workspace in the Name box.
    4. For Region, select the Azure cloud region where the workspace will be deployed.
    5. For Storage account, choose the storage account that is used as the default datastore for the workspace.
    6. For Key vault, select the key vault used to store secrets and other sensitive information required by the workspace.
    7. For Application insights, choose whether to enable the Application Insights monitoring feature. You can create a new Azure Application Insights resource or select an existing one from your subscription.
    8. For Container registry, select the container registry used to register docker images used in training and deployments.
    9. Choose Next : Networking to continue the workspace setup process.
  2. For Networking, perform the following actions:
    1. For Network isolation, select Private with Internet Outbound for the type of network isolation required for your workspace.
    2. For Workspace Outbound access, choose Use a virtual network managed by Azure Machine Learning to enable network isolation with an Azure managed virtual network (managed VNet).
    3. Choose Add user-defined outbound rules and perform the following operations:
      1. For Rule name, provide a unique name for the new outbound rule.
      2. For Destination type, choose Private Endpoint. Private Endpoint is the only option when the managed virtual network isolation is private with Internet outbound.
      3. For Subscription, select the subscription that contains the Azure cloud resource you want to add a private endpoint for.
      4. For Resource group, choose the resource group that contains the cloud resource you want to add a private endpoint for.
      5. For Resource type, select the type of the cloud resource for the private endpoint (Azure Storage account or Azure key vault).
      6. For Resource name, choose the name of the target resource for the private endpoint.
      7. For Sub resource, select the sub resource to connect for the private endpoint.
      8. (Optional) Check the Spark enabled setting checkbox if you want to enable serverless Spark jobs for your workspace. This option is only available if the Resource type is Microsoft.Storage/storageAccounts.
      9. Choose Save to deploy the new outbound rule. This rule defines private endpoints that allow an Azure resource such as a storage account or key vault to securely communicate with the managed VNet.
    4. Choose Next : Encryption to continue the setup.
  3. For Encryption, perform the following actions:
    1. For Encryption type, choose Customer-managed keys to use your own Customer-Managed Key (CMK) for data encryption (recommended).
    2. For Key vault, choose Click to select the key, and select the appropriate key vault and Customer-Managed Key (CMK).
    3. Choose Next : Identity to continue.
  4. For Identity, perform the following operations:
    1. Choose System assigned identity for Identity type under Managed identity.
    2. Select Identity-based access for Storage account access type under Storage account access.
    3. Under Data impact, check the High business impact workspace setting checkbox if you want to enable the High Business Impact (HBI) feature for your workspace.
    4. Choose Next : Tags to continue the setup.
  5. For Tags, use the Name and Value fields to create tags that will help organize the identity of the workspace. Choose Next : Review + create to validate the workspace setup.
  6. For Review + create, review the resource configuration details, then choose Create to create your new Azure Machine Learning workspace.

05 Repeat step no. 4 for each workspace that you want to re-create, available within the selected Azure subscription.

06 Repeat steps no. 3 – 5 for each subscription available in your Microsoft Azure cloud account.

Using Azure CLI

01 Define the configuration file for your new Azure Machine Learning workspace. Create a new configuration file (YAML format), name the file workspace-config.yml, and paste the following content. Set isolation_mode to allow_internet_outbound to enable managed virtual network isolation with Internet outbound access and use the outbound_rules parameter to define user-defined outbound rules. The following rule example demonstrates adding a private endpoint to an Azure Storage blob resource. Replace other configuration parameter values with your own information:

$schema: https://azuremlschemas.azureedge.net/latest/workspace.schema.json
name: tm-new-project5-workspace
location: westeurope
display_name: tm-new-project5-workspace
identity:
type: system_assigned
managed_network:
isolation_mode: allow_internet_outbound
outbound_rules:
- name: blob-access-rule
	destination:
	service_resource_id: /subscriptions/abcd1234-abcd-1234-abcd-1234abcd1234/resourceGroups/cloud-shell-storage-westeurope/providers/Microsoft.Storage/storageAccounts/1234abcd12341234abcd1234
	spark_enabled: true
	subresource_target: blob
	type: private_endpoint

02 Run ml workspace create command (Windows/macOS/Linux) with the configuration file defined at the previous step (i.e. workspace-config.yml), to create your new Azure Machine Learning workspace:

az ml workspace create
  --resource-group cloud-shell-storage-westeurope
  --file workspace-config.yml

03 The command output should return the new workspace configuration information:

{
	"enable_data_isolation": false,
	"description": "tm-new-project5-workspace",
	"discovery_url": "https://westeurope.api.azureml.ms/discovery",
	"display_name": "tm-new-project5-workspace",
	"id": "/subscriptions/abcd1234-abcd-1234-abcd-1234abcd1234/resourceGroups/cloud-shell-storage-westeurope/providers/Microsoft.MachineLearningServices/workspaces/tm-new-project5-workspace",
	"identity": {
		"principal_id": "abcd1234-abcd-1234-abcd-1234abcd1234",
		"tenant_id": "abcd1234-abcd-1234-abcd-1234abcd1234",
		"type": "system_assigned"
	},
	"key_vault": "/subscriptions/abcd1234-abcd-1234-abcd-1234abcd1234/resourceGroups/cloud-shell-storage-westeurope/providers/Microsoft.Keyvault/vaults/1234abcd12341234abcd1234",
	"location": "westeurope",
	"managed_network": {
		"isolation_mode": "allow_internet_outbound",
		"network_id": "abcd1234-abcd-1234-abcd-1234abcd1234",
		"outbound_rules": [
			{
				"category": "user_defined",
				"destination": {
					"service_resource_id": "/subscriptions/abcd1234-abcd-1234-abcd-1234abcd1234/resourceGroups/cloud-shell-storage-westeurope/providers/Microsoft.Storage/storageAccounts/1234abcd12341234abcd1234",
					"spark_enabled": true,
					"subresource_target": "blob"
				},
				"name": "blob-access-rule",
				"status": "Inactive",
				"type": "private_endpoint"
			}
		],
		"status": {
			"spark_ready": false,
			"status": "Inactive"
		}
	},
	"mlflow_tracking_uri": "azureml://westeurope.api.azureml.ms/mlflow/v1.0/subscriptions/abcd1234-abcd-1234-abcd-1234abcd1234/resourceGroups/cloud-shell-storage-westeurope/providers/Microsoft.MachineLearningServices/workspaces/tm-new-project5-workspace",
	"name": "tm-new-project5-workspace",
	"public_network_access": "Disabled",
	"resourceGroup": "cloud-shell-storage-westeurope",
	"resource_group": "cloud-shell-storage-westeurope",
	"serverless_compute": {
		"no_public_ip": false
	},
	"storage_account": "/subscriptions/abcd1234-abcd-1234-abcd-1234abcd1234/resourceGroups/cloud-shell-storage-westeurope/providers/Microsoft.Storage/storageAccounts/1234abcd12341234abcd1234",
	"tags": {
		"AttachAppInsightsToWorkspace": "false",
		"createdByToolkit": "cli-v2-1.16.1"
	}
}

04 Repeat steps no. 1 - 3 for each workspace that you want to re-create, available in the selected Azure subscription.

05 Repeat steps no. 1 – 4 for each subscription available in your Microsoft Azure cloud account.

References

Publication date Jun 19, 2024