- Knowledge Base
- Google Cloud Platform
- GCP VertexAI
- Default VPC Network In Use
Ensure that your Google Cloud Vertex AI notebook instances are not created within the default Virtual Private Cloud (VPC) network in order to follow security best practices and meet networking requirements.
This rule resolution is part of the Conformity Security & Compliance tool for GCP.
A default Virtual Private Cloud (VPC) is designed in such a way that you can quickly deploy GCP resources and not have to think about the underlying network. The default VPC comes with a predefined network configuration that automatically generates 4 over-permissive, insecure firewall rules, that are not included in the audit logging:
- "default-allow-internal" – this rule allows ingress connections for all TCP, UDP and ICMP protocols and all ports (0-65535) among VM instances within the network.
- " default-allow-ssh" – allows ingress connections on TCP port 22 (SSH) from any source to any virtual machine (VM) instance in the network.
- "default-allow-rdp" – this firewall rule allows ingress connections on TCP port 3389 (RDP) from any source to any VM instance in the network.
- "default-allow-icmp" – allows ingress ICMP traffic from any source to any VM instance within the network.
The default Virtual Private Cloud (VPC) network is also an auto-mode network, which means that its subnets use the same predefined range of IPv4 addresses, and as a result, it's not possible to use Cloud VPN or VPC Network Peering feature with the default network. While suitable for quick starts with Vertex AI instances, complex production AI applications with multi-tier architectures may require private network segments or customization. To address these needs, consider using a non-default VPC tailored to your application requirements for Vertex AI notebook instances.
Audit
To determine if the default VPC network is used by your Vertex AI notebook instances, perform the following operations:
Using GCP Console
01 Sign in to the Google Cloud Management Console.
02 Select the GCP project that you want to examine from the console top navigation bar.
03 Navigate to Vertex AI console available at https://console.cloud.google.com/vertex-ai.
04 In the main navigation panel, under NOTEBOOKS, choose Workbench, and select the INSTANCES tab.
05 Choose View: INSTANCES to list the Vertex AI notebook instances created for the selected GCP project.
06 Click on the name (link) of the notebook instance that you want to examine.
07 Select the SYSTEM tab and click on the name (link) of the VPC subnet configured for the selected instance, listed next to Subnetwork.
08 On the subnet page, check the VPC Network attribute value to identify the name of the subnet's network. If the VPC Network value is default, the selected Vertex AI notebook instance is using the default Virtual Private Cloud (VPC) network created for the selected GPC project.
09 Repeat steps no. 6 - 8 for each Vertex AI notebook instance launched for the selected GCP project.
10 Repeat steps no. 2 – 9 for each project deployed within your Google Cloud account.
Using GCP CLI
01 Run projects list command (Windows/macOS/Linux) with custom query filters to list the ID of each project available in your Google Cloud account:
gcloud projects list --format="table(projectId)"
02 The command output should return the requested GCP project ID(s):
PROJECT_ID cc-vertex-project-123123 cc-appdata-project-112233
03 Run workbench instances list command (Windows/macOS/Linux) with the ID of the GCP project that you want to examine as the identifier parameter, to describe the name of each Vertex AI notebook instance created for the selected project:
gcloud workbench instances list --project cc-vertex-project-123123 --location=us-central1-a --format="(NAME)"
04 The command output should return the requested notebook instance names:
NAME: tm-vertex-ai-notebook-instance NAME: tm-development-notebook-instance
05 Run workbench instances describe command (Windows/macOS/Linux) with the name of the Vertex AI notebook instance that you want to examine as the identifier parameter and custom output filters to describe the identifier of the VPC network used by the selected instance:
gcloud workbench instances describe tm-vertex-ai-notebook-instance --location=us-central1-a --format="value(gceSetup.networkInterfaces[].network)"
06 The command output should return identifier (full URI) of the associated VPC network. The URI has the following format: <https://www.googleapis.com/compute/v1/projects/\><project-id\>/global/networks/\<vpc-name\>
, where \<project-id\> is the GCP project ID and the \<vpc-name\> is the name of the VPC network:
https://www.googleapis.com/compute/v1/projects/cc-vertex-project-123123/global/networks/default
If the URI returned by the workbench instances describe command output ends with default, as shown in the output example above, the selected Vertex AI notebook instance is using the default Virtual Private Cloud (VPC) network available for the selected GPC project.
07 Repeat steps no. 5 and 6 for each Vertex AI notebook instance provisioned for the selected GCP project.
08 Repeat steps no. 3 – 7 for each GCP project deployed in your Google Cloud account.
Remediation / Resolution
To deploy your Google Cloud Vertex AI notebook instances to a custom, non-default VPC network, you must re-create your instances with the appropriate network configuration, by performing the following operations:
Using GCP Console
01 Sign in to the Google Cloud Management Console.
02 Select the GCP project that you want to access from the console top navigation bar.
03 Navigate to VPC network console available at https://console.cloud.google.com/networking.
04 In the main navigation panel, select VPC networks, and choose CREATE VPC NETWORK from the console top menu to initiate the VPC setup process.
05 On the Create a VPC network setup page, perform the following actions:
- For Name, enter a unique name for your new VPC network.
- (Optional) Provide a short description for the network in the Description text box.
- (Optional) Select the maximum transmission unit (MTU) from the Maximum transmission unit (MTU) dropdown list.
- For Subnet creation mode, choose Custom to manually define the VPC network subnets. The custom mode network type provides you with complete control over its subnets and IPv4 address ranges. You decide which subnets to create within Google Cloud regions that you choose by using IPv4 ranges that you specify.
- (Optional) For Private IPv6 address settings, you can configure a ULA internal IPv6 range for the new VPC network.
- For Subnets, choose ADD SUBNET, and provide the following configuration parameters:
- Provide a name and a description for the new subnet in the Name and Description boxes.
- Select the GCP region where you want to deploy your subnet, from the Region dropdown list.
- For IP stack type, select the IP address stack type that you want to use for your subnet.
- In the IPv4 range box, enter the IPv4 range for this subnet, in CIDR notation. You can use a standard private VPC network address range such as 10.0.0.0/9. This is the primary IPv4 range for this subnet. (Optional) To define a secondary IPv4 range for this subnet, choose CREATE SECONDARY IPV4 RANGE and provide the secondary IP range (CIDR notation) and a name for the secondary subnet range.
- For Private Google Access, set whether virtual machines (VMs) created in this subnet can access Google Cloud services without assigning external IP addresses.
- (Optional) For Flow logs, choose whether to enable the VPC Flow Logs feature at subnet creation or enable it later by editing the VPC subnet configuration.
- (Optional) For Hybrid subnet, choose whether to make the new subnet a hybrid subnet. Enabling hybrid subnets alters the VPC network routing behavior to permit overlap between the subnet's IP address range and custom dynamic route address ranges.
- Choose DONE to create the custom VPC subnet.
- To create more custom subnets for your VPC network, choose ADD SUBNET and follow the setup wizard.
- For Firewall rules, select any existing firewall rules you would like to add to the VPC configuration. You can add additional rules after the VPC is created.
- For Advanced dynamic routing configuration, choose whether to use Regional or Global dynamic routing for your new VPC network. The Regional dynamic routing mode (default) uses Cloud Routers to learn routes only in the region in which they were created. If you are using an internal load balancer with a dedicated interconnect or a VPN on this VPC network, use the Regional dynamic routing. The Global dynamic routing mode lets you dynamically learn routes to and from all GCP regions with a single VPN or dedicated interconnect, and a Cloud Router.
- (Optional) For DNS configuration (optional), select an existing DNS server policy from the DNS server policy dropdown list or choose to create a new one. You can have only one DNS server policy for each VPC network within your Google Cloud account. The DNS server policy can specify inbound forwarding, outbound forwarding, or both.
- Click CREATE to deploy your new, non-default Virtual Private Cloud (VPC) network.
06 Once the new VPC network is created, click on the VPC name, select the FIREWALLS tab, and use the ADD FIREWALL RULE button to create firewall rules that allow or deny traffic between the resources inside the network, such as communication between virtual machine (VM) instances. You can also use network firewall rules to control what traffic leaves or enters the VPC network to and from the Internet.
07 Navigate to Vertex AI console at https://console.cloud.google.com/vertex-ai.
08 In the main navigation panel, under NOTEBOOKS, choose Workbench, and select the INSTANCES tab.
09 Choose CREATE NEW, select ADVANCED OPTIONS, and perform the following actions to create your new notebook instance:
- For Details, provide the following information:
- For Name, enter a unique name for your new notebook instance.
- For Region and Zone, select the GCP location where the instance will be deployed.
- (Optional) Check the Enable Dataproc Serverless Interactive Sessions setting checkbox to enable access to Dataproc Spark kernels.
- (Optional) For Labels, choose ADD LABEL, and use the Key and Value fields to create labels for the new instance.
- (Optional) Use Network tags to assign network tags to your Workbench instance.
- For Workbench type, choose Instance.
- Choose Continue to continue the instance setup.
- For Environment, perform the following actions:
- Choose whether to use a custom container or the latest version of the Vertex AI Workbench for the instance environment.
- (Optional) For Post-startup script, you can select a script that automatically runs after the instance boots up.
- (Optional) For Metadata, choose ADD METADATA to add metadata keys to your Workbench instance.
- Choose Continue to continue the setup.
- For Machine type, perform the following operations:
- For Machine type, choose the appropriate machine type for your workload.
- For Shielded VM, check the Secure Boot, Virtual Trusted Platform Module (vTPM), and Integrity monitoring checkboxes for the most secure instance configuration.
- For Idle shutdown, check the Enable Idle Shutdown checkbox to enable the Idle Shutdown feature for the new instance. Enter the preferred idle timeout value (in minutes) in the Time of inactivity before shutdown (Minutes) box.
- Choose Continue to continue the setup process.
- For Disks, perform the following operations:
- For Disks, choose the boot disk type and boot disk size (GB) for the instance disks. (Optional) Check the Delete to trash checkbox if you want to use the operating system's trash behavior.
- For Encryption, choose Cloud KMS key, and select the Cloud KMS Customer-Managed Encryption Key (CMEK) that you want to use for data encryption (recommended).
- Choose Continue to continue the setup.
- For Networking, choose Network in this project, and select the custom VPC network and subnetwork created at step no. 5. Choose whether to allow HTTPS access to your JupyterLab instance. For network isolation and stringent compliance, uncheck the Assign external IP address checkbox to prevent adding an external IP address to the instance. Choose Continue to continue the setup process.
- For IAM and security, perform the following actions:
- For IAM and security, configure who can use the instance's JupyterLab interface. Choose Service account for default instance access or choose Single user to restrict access to one user only. Choose whether to use the default Compute Engine service account or a custom service account.
- For Security options, uncheck the Root access to the instance checkbox to disable the root access to the new instance, and choose whether to allow terminal access and file downloads from JupyterLab.
- Choose Continue to continue the setup.
- For System health, perform the following operations:
- For System health, check the Environment auto-upgrade checkbox to enable automatic upgrades. Choose whether to upgrade your new instance Weekly or Monthly.
- For Reporting, check the Install Cloud Monitoring checkbox to install the Cloud Monitoring agent and enable the Cloud Monitoring feature. You can also check the Report custom metrics to Cloud Monitoring checkbox to collect system status and JupyterLab metrics. Ensure that Report system health and Report DNS status for required Google domains checkboxes are also checked for core service and DNS status verification.
- Choose CREATE to launch your new Google Cloud Vertex AI notebook instance.
10 Repeat step no. 9 for each Vertex AI notebook instance that you want to re-create, launched for the selected GCP project.
11 Repeat steps no. 2 – 10 for each project deployed within your Google Cloud account.
Using GCP CLI
01 Run compute networks create command (Windows/macOS/Linux) to create a custom (non-default) Virtual Private Cloud (VPC) network within the GCP project referenced as value for the --project parameter:
gcloud compute networks create custom-vpc-network --project cc-vertex-project-123123 --subnet-mode=custom --bgp-routing-mode=regional
02 The command output should return the configuration information available for the newly created VPC network (including the full URI of the network):
Created [https://www.googleapis.com/compute/v1/projects/cc-vertex-project-123123/global/networks/custom-vpc-network]. NAME SUBNET_MODE BGP_ROUTING_MODE IPV4_RANGE GATEWAY_IPV4 custom-vpc-network CUSTOM REGIONAL Instances on this network will not be reachable until firewall rules are created. As an example, you can allow all internal traffic between instances as well as SSH, RDP, and ICMP by running: $ gcloud compute firewall-rules create <FIREWALL_NAME> --network custom-vpc-network --allow tcp,udp,icmp --source-ranges <IP_RANGE> $ gcloud compute firewall-rules create <FIREWALL_NAME> --network custom-vpc-network --allow tcp:22,tcp:3389,icmp
03 Run compute networks subnets create command (Windows/macOS/Linux) to create and attach a custom subnet to the VPC network created at the previous steps. The following command example creates a VPC network subnet named "us-central1-subnet", in the Iowa, US (us-central1) region with the primary IP address range set to 10.0.0.0/24. Use the compute networks subnets create command to create as many VPC subnets as you want:
gcloud compute networks subnets create us-central1-subnet --network=custom-vpc-network --range=10.0.0.0/24 --region=europe-west2
04 The command output should return the VPC subnet configuration information (including the URI of the subnet):
Created [https://www.googleapis.com/compute/v1/projects/cc-vertex-project-123123/regions/us-central1/subnetworks/us-central1-subnet]. NAME REGION NETWORK RANGE us-central1-subnet us-central1 custom-vpc-network 10.0.0.0/24
05 Run compute firewall-rules create command (Windows/macOS/Linux) to create the necessary firewall rules for your new, non-default Virtual Private Cloud (VPC) network. Firewall rules control incoming and/or outgoing traffic to GCP resources such as VM instances. The following command example creates a firewall rule that allows inbound traffic to all VM instances deployed within the VPC network through TCP port 8888:
gcloud compute firewall-rules create allow-custom-tcp-traffic --network custom-vpc-network --allow tcp:8888 --direction ingress --source-ranges 0.0.0.0/0 --enable-logging
06 The command output should return the VPC firewall rule configuration information:
Created [https://www.googleapis.com/compute/v1/projects/cc-vertex-project-123123/global/firewalls/allow-custom-tcp-traffic]. NAME NETWORK DIRECTION PRIORITY ALLOW DENY DISABLED allow-custom-tcp-traffic custom-vpc-network INGRESS 1000 tcp:8888 False
07 Run workbench instances create command (Windows/macOS/Linux) to deploy your Google Cloud Vertex AI notebook instance to the custom, non-default VPC network created and configured at the previous steps. Use the --network and --subnet command parameters to specify the custom VPC network and subnetwork that you want to use for your new instance:
gcloud workbench instances create tm-vertex-ai-notebook-instance --project=cc-vertex-project-123123 --container-repository=gcr.io/deeplearning-platform-release/base-cpu --container-tag=latest --machine-type=e2-standard-2 --location=us-central1-a --shielded-integrity-monitoring=true --shielded-secure-boot=true --shielded-vtpm=true --network=https://www.googleapis.com/compute/v1/projects/cc-vertex-project-123123/global/networks/custom-vpc-network --subnet=https://www.googleapis.com/compute/v1/projects/cc-vertex-project-123123/regions/us-central1/subnetworks/us-central1-subnet --format="yaml(gceSetup.networkInterfaces[].network)"
08 The command output should return the return identifier (full URI) of the custom VPC network associated with the notebook instance:
Waiting for operation on Instance [tm-vertex-ai-notebook-instance] to be updated with [projects/cc-vertex-project-123123/locations/us-central1-a/operations/operation-abcd1234abcd-abcd1234abcd-abcd1234-abcd1234]...done. Created workbench instance tm-vertex-ai-notebook-instance [https://notebooks.googleapis.com/v2/projects/cc-vertex-project-123123/locations/us-central1-a/operations/operation-abcd1234abcd-abcd1234abcd-abcd1234-abcd1234]. https://www.googleapis.com/compute/v1/projects/cc-vertex-project-123123/global/networks/custom-vpc-network
09 Repeat steps no. 7 and 8 for each Vertex AI notebook instance that you want to re-create, provisioned for the selected GCP project.
10 Repeat steps no. 1 – 9 for each GCP project deployed in your Google Cloud account.
References
- Google Cloud Platform (GCP) Documentation
- Introduction to Vertex AI Workbench
- Vertex AI Workbench Jupyter Notebook tutorials
- Create a Vertex AI Workbench instance