Create VMs with multiple network interfaces

By default, every virtual machine (VM) instance in a Virtual Private Cloud (VPC) network has a single network interface. However, you can configure an instance with multiple network interfaces. When using multiple network interfaces from an instance, each interface must attach to a subnet that is in a different VPC network. You can't attach multiple network interfaces to the same subnet or to subnets that are in the same VPC network.

You can configure VM interfaces with IPv4 addresses only (single-stack) or both IPv4 and IPv6 addresses (dual-stack).

If you don't require multiple network interfaces, follow the procedure for Creating and starting an instance to create instances with the default network configuration.

For more information about multiple network interfaces and how they work, see Multiple network interfaces.

Specifications

You can only configure a network interface when you create an instance.
You can't configure multiple network interfaces for Compute Engine bare metal instances.
If you create an instance with multiple network interfaces, each interface must connect to a different VPC network. The networks can be standalone VPC networks or Shared VPC networks.
A network interface must connect to a subnet that has an IP address range that does not overlap with the IP address range of another interface on the same instance.
If you want to create a VM that is attached to subnets in multiple VPC networks, all of the networks and subnets must exist before you create the instance. For more information about creating networks and subnets, see Create and manage VPC networks.
You can't delete a network interface without deleting the instance that it is attached to.
Each instance must have a minimum of one network interface and a maximum of eight, depending on the instance's machine type:
- nic0 is required and must be configured at instance creation.
- nic1 to nic7 are optional, but must also be configured at instance creation.
For more information, see Maximum number of interfaces.
Every interface can optionally have an external IPv4 address.
If you connect a VM to a subnet that has an internal or external IPv6 subnet range, you can optionally configure an internal or external IPv6 address.
Google Cloud's DHCP server sends a default route (RFC 3442, "classless static route") only to the default network interface, nic0. If a default route is needed on another network interface, you must configure policy routing on the instance. For an example of how to do this, see the following tutorial: Configure routing for an additional interface.
All VM network interfaces must connect to subnets in networks that are in the same project as the VM or to a Shared VPC network that is shared with the project.

Limitations

You cannot add or remove network interfaces from an existing VM.
IP forwarding is enabled at the VM level and cannot be applied to individual interfaces.

Permissions, IAM, and instances with multiple network interfaces

To create an instance with multiple network interfaces, you must have one of the following roles:

Project Owner or Editor role
Compute Instance Admin (v1) role (roles/compute.instanceAdmin.v1)

Creating and deleting instances and instance templates with multiple interfaces in a project not using a Shared VPC environment: A user with the Owner, Editor, or Compute Instance Admin (v1) role at the project level can create an instance with multiple interfaces associated with VPC networks and subnets that are part of that same project.

Creating and deleting instances and instance templates with multiple interfaces in Shared VPC environments: A user with the Owner, Editor, or Compute Instance Admin (v1) role at the project level can create an instance with multiple interfaces. If any of the interfaces are attached to a subnet in a Shared VPC host project, you must also have the Compute Network User role (roles/compute.networkUser) at the Shared VPC host project level or at the Shared VPC subnet level.

To learn more about permissions, read the Compute Engine IAM documentation.

Use multiple network interfaces with different VPC network types

Multiple network interfaces are not supported with legacy networks.
When you configure multiple network interfaces, attach each interface to an auto mode VPC network or custom mode VPC network.

Network interface IP address allocation

You must configure each network interface with a primary internal IP address that is allocated from the primary IP address range of a subnet.
- The primary internal address allocated to each of the interfaces in a single instance must be different.
You can choose to configure each of the virtual interfaces (NIC) with a unique external IP address. External addresses can be ephemeral or reserved.

Maximum number of network interfaces

The number of virtual network interfaces scales with the number of vCPUs with a minimum of 2 and maximum of 8.

Use the following table to determine how many network interfaces can be attached to an instance:

Number of vCPU	Number of vNICs
2 or less	2
4	up to 4
6	up to 6
8 or more	up to 8
Compute Engine bare metal instances	1

Before you begin

Verify that all required networks and subnets are created before you create the instance. If you need to create networks or subnets, see Create and manage VPC networks.
Verify that each network has appropriate firewall rules to allow the traffic that you want to allow to and from the VM that has multiple interfaces. If you need to create firewall rules, see Use VPC firewall rules.
If you are connecting a VM to multiple networks by using IPv6 addresses, install google-guest-agent version 20220603.00 or later. For more information, see I can't connect to a secondary interface's IPv6 address.

Create VM instances with multiple network interfaces

For general instructions about creating instances, see Create and start a VM instance.

You can add multiple network interfaces when you create an instance:

In the Google Cloud console, you add network interfaces on the Create an Instance page, in the Networking > Network interfaces section.
In the gcloud CLI, use the instances create command. Include the --network-interface flag for each interface, followed by any appropriate networking keys, such as ([network | subnet], private-network-ip, address, external-ipv6-address).

The first interface is always created as nic0, and it is always the default interface. This is important for some other aspects of Google Cloud networking. For example, Google Cloud load balancers (except for the passthrough Network Load Balancers) only distribute traffic to nic0.

Console

In the Google Cloud console, go to the Create an instance page.

Go to Create an instance
In the Name field, enter a name for the instance.
In the Region field, select a region.
In the Zone field, select a zone.
In the Advanced options section, expand Networking, and then do the following:
1. In the Network interfaces section, expand the network interface to edit it.
2. For Network and Subnetwork, select the network and subnetwork that you want to use.
If you want to configure IPv6 addresses on the interface, select a subnet that has an IPv6 address range configured. The IPv6 access type of the subnet determines whether the VM receives an internal IPv6 address or an external IPv6 address.
1. Select an IP stack type: IPv4 (single-stack) or IPv4 and IPv6 (dual-stack)
2. For Primary internal IPv4 address, select one of the following:
  - Ephemeral to assign a new ephemeral IPv4 address
  - A reserved static internal IPv4 address from the list
  - Reserve static internal IPv4 address to reserve and assign a new static internal IPv4 address
3. For Primary internal IPv6 address, select one of the following:
  - Auto-allocated to automatically assign a new ephemeral internal IPv6 address
  - A reserved static internal IPv6 address from the list
  - Reserve static internal IPv6 address to reserve and assign a new static internal IPv6 address
4. For External IPv4 address, select one of the following:
  - Ephemeral to assign a new ephemeral IPv4 address
  - None to not assign an external IPv4 address
  - A reserved static IPv4 address from the list
  - Reserve static external IP address to reserve and assign a new static external IPv4 address
5. For External IPv6 address, select one of the following:
  - Auto-Allocate to automatically assign a new ephemeral external IPv6 address
  - A reserved static external IPv6 address from the list
  - Reserve static external IPv6 address to reserve and assign a new static external IPv6 address
6. To finish modifying the network interface, click Done.
To add another interface, click Add network interface.
Continue with the VM creation process.
Click Create.

gcloud

To create network interfaces on a new instance, use the instances create command.

Include the --network-interface flag for each interface, followed by any appropriate networking keys, such as network, subnet, private-network-ip, address, and external-ipv6-address. To view examples of how to create VMs with multiple interfaces, see Example configurations.

This snippet only illustrates the --network-interface flag, one of many possible parameters that you can specify when creating an instance.

To see what machine types support the number of network interfaces you need, see the Maximum number of network interfaces table.

gcloud compute instances create INSTANCE_NAME \
    --zone ZONE \
    --network-interface \
        network=NETWORK,subnet=SUBNET, \
        stack-type=STACK_TYPE, \
        address=EXTERNAL_IPV4_ADDRESS | no-address, \
        private-network-ip=INTERNAL_IPV4_ADDRESS \
        internal-ipv6-address=INTERNAL_IPV6_ADDRESS \
    ...
    --network-interface \
        network=NETWORK,subnet=SUBNET, \
        stack-type=STACK_TYPE, \
        external-ipv6-address=EXTERNAL_IPV6_ADDRESS, \
        external-ipv6-prefix-length=96 \
        ipv6-network-tier=PREMIUM \
    ...

Replace the following:

INSTANCE_NAME: the name of the VM instance to create.
ZONE: the zone where the instance is created.
NETWORK: the network where the interface attaches.
SUBNET: the subnet where the interface attaches.
STACK_TYPE: the stack type for the interface.

The default value is IPV4_ONLY. Specify IPV4_IPV6 to configure a dual-stack interface.
EXTERNAL_IPV4_ADDRESS: assigns the named external IPv4 address to the interface.

You must have previously reserved an external IPv4 address. Specify "no-address" instead of address=EXTERNAL_IPV4_ADDRESS if you don't want the interface to have an external IP address. Specify address='' if you want the interface to receive an ephemeral external IP address.
INTERNAL_IPV4_ADDRESS: the internal IPv4 address that you want the interface to have in the target subnet. Omit if you just want any valid address assigned.
INTERNAL_IPV6_ADDRESS: the internal IPv6 address that you want the interface to have in the target subnet. If you omit this flag, an internal IPv6 address is not assigned.
EXTERNAL_IPV6_ADDRESS: assigns the named external IPv4 address to the interface.

You must have previously reserved an external IPv6 address. Specify "no-address" instead of external-ipv6-address=EXTERNAL_IPV6_ADDRESS if you don't want the interface to have an external IPv6 address. Specify external-ipv6-address='' if you want the interface to receive an ephemeral external IPv6 address.

Note: An interface can have only one IPv6 address, either internal or external, but not both.

API

Use the instances.insert method to create a VM instance with multiple network interfaces.

To create a VM instance with internal IPv4 addresses, do the following:
```
POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances
{
....

"networkInterfaces": [
  {
    "networkIP": "IPV4_ADDRESS",
    "subnetwork": "regions/REGION/subnetworks/SUBNETWORK"
  },
    for each interface, specify a network...
],
other instance settings...
}
```
Replace the following:
- SUBNET: the subnet where the network interface is located.
- REGION: the region that contains the instance.
- PROJECT_ID: the ID of the project that contains the instance.
- ZONE: the zone that contains the instance.
- IPV4_ADDRESS: the internal IPv4 address that you want to assign to the network interface.

To create a VM instance with internal IPv6 addresses, do the following:

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances
{
....

"networkInterfaces": [
  {
    "subnetwork": "regions/REGION/subnetworks/SUBNETWORK"
    "ipv6Address": "IPV6_ADDRESS",
    "internalIpv6PrefixLength": 96,
    "stackType": IPV4_IPV6,
    "ipv6AccessType": INTERNAL
  },
    for each interface, specify a network...
],
other instance settings...
}

Replace the following:

SUBNET: the subnet where the network interface is located.
REGION: the region that contains the instance.
PROJECT_ID: the ID of the project that contains the instance.
ZONE: the zone that contains the instance.
IPV6_ADDRESS: the internal IPv6 address that you want to assign to the network interface.

Terraform

You can use a Terraform resource to create a VM instance with multiple network interfaces.

The Terraform arguments have example values that you can change.

resource "google_compute_instance" "default" {
  project      = var.project_id # Replace with your project ID in quotes
  zone         = "us-central1-b"
  name         = "backend-instance"
  machine_type = "e2-medium"
  boot_disk {
    initialize_params {
      image = "debian-cloud/debian-9"
    }
  }
  network_interface {
    subnetwork = var.subnet_1 # Replace with self link to a subnetwork in quotes
    network_ip = "10.0.0.14"
  }
  network_interface {
    subnetwork = var.subnet_2 # Replace with self link to a subnetwork in quotes
    network_ip = "10.10.20.14"
  }
}

To learn how to apply or remove a Terraform configuration, see Basic Terraform commands.

Example configurations

The following sections show how to create VMs with multiple interfaces.

Configure multiple network interfaces with IPv4 addresses

The following sample command creates an instance with three network interfaces.

 gcloud compute instances create vm1 --machine-type=n1-standard-4 \
    --network-interface '' \
    --network-interface network=net1,subnet=subnet-a,private-network-ip=10.10.10.2,address=EXTERNAL_IPV4_ADDRESS \
    --network-interface network=net2,subnet=subnet-b,private-network-ip=10.10.20.2,no-address

The interfaces are created as follows:

nic0 is created with default settings. The interface is attached to a subnet in the default VPC network, with an automatically allocated internal IP address and an ephemeral external IP address.
nic1 is attached to subnet subnet-a in network net1, with an internal IPv4 address of 10.10.10.2 and a static external IPv4 address, EXTERNAL_IPV4_ADDRESS.
nic2 is attached to subnet subnet-b in network net2, with an internal IPv4 address of 10.10.20.2 and no external IP address.

For a complete description of the gcloud compute instances create command and the --network-interface flag, read the documentation for the command.

You can use the IP address from the network interface that you have added to set up DNS forwarding. To learn more about configuring Cloud DNS forwarding zones, see Forwarding zones.

Configure multiple network interfaces with IPv4 and IPv6 addresses

The following sample command creates a dual-stack instance with two network interfaces.

gcloud compute instances create vm1 \
--network-interface network=dual-int,subnet=int-subnet,stack-type=IPV4_IPV6 \
--network-interface network=dual-ext,subnet=ext-subnet,stack-type=IPV4_IPV6,ipv6-network-tier=PREMIUM \
--machine-type=n1-standard-4 --zone=us-west2-a

The interfaces are created as follows:

nic0 is attached to subnet int-subnet in network dual-int, with an ephemeral internal IPv4 address and an ephemeral internal IPv6 address.
nic1 is attached to subnet ext-subnet in network dual-ext, with an ephemeral internal IPv4 address and an ephemeral external IPv6 address.

Configure multiple network interfaces for instance groups

You can use instances with multiple network interfaces in unmanaged instance groups and managed instance groups.

For unmanaged instance groups, create each instance individually, ensuring that the nic0 network interface for each VM is attached to the same subnet. Then, add the VM instances to the unmanaged instance group.

To configure multiple network interfaces for managed instance groups, you must specify the network configuration for each interface in the instance template, by setting the --network-interface flag once for each interface. The following sample creates an instance template with three network interfaces:

gcloud compute instance-templates create template-1 \
    --network-interface subnet=net0-subnet-a \
    --network-interface subnet=net1-subnet-b,no-address \
    --network-interface subnet=net2-subnet-c,no-address \
    --region us-central1

Because subnet names in each region of a project must be unique, specifying subnets by name implicitly associates each interface with a VPC network. Each interface must use a subnet that is in a unique VPC network:

nic0 uses the net0-subnet-a subnet
nic1 uses the net1-subnet-b subnet
nic2 uses the net2-subnet-c subnet

The no-address option in the --network-interface flag indicates that the interface is configured without an external IP address. The internal IP address comes from the subnet used by the interface. For complete information about the flags and syntax, see the --network-interface flag for the instance-templates create command.

Troubleshooting

I can't create a VM with multiple interfaces

You may see one of the following error messages:

Invalid value for field 'resource': ''. Too many network interfaces. The maximum number of network interfaces allowed for this machine type is.

If you get this error message, you are trying to create more interfaces than the maximum interfaces supported by your instance machine type. See the maximum number of interfaces table.
Networks must be distinct for NICs attached to a VM.

If you get this message, you are trying to create more than one interface in the same network. Each network interface must be attached to a different VPC network.
Subnetwork CIDR ranges must be non-overlapping for NICs attached to a VM.

If you get this message, the CIDR ranges associated with your VM's interfaces are overlapping. These CIDR ranges include all primary ranges of subnets associated with VM interfaces as well as secondary ranges used for alias IP ranges. Each interface is attached to a subnet, each in a different VPC network, that must not overlap with subnets of other interfaces. For example, if you are trying to create your instance in region us-west1, you can check subnet primary CIDR ranges using the following command or the Google Cloud console.
```
gcloud compute networks subnets list --regions us-west1
NAME                REGION    NETWORK          RANGE
default             us-west1  default          10.138.0.0/20
overlapping-subnet  us-west1  test-network     10.138.8.0/24
```
To check secondary subnet CIDR ranges, use the following command or the Google Cloud console.
```
gcloud compute networks subnets describe overlapping-subnet --region us-west1
```
```
...
ipCidrRange: 10.128.8.0/24
...
secondaryIpRanges:
- ipCidrRange: 10.138.8.0/24
  rangeName: conflicting-range
```
Note: secondary ranges present in a subnet associated with VM interface but not used by the VM interface are not checked for overlap.
Multiple network interfaces are not supported on legacy networks.

If you get this message, you are trying to create your instance in a legacy network. Instances with multiple interfaces are not supported with legacy networks. You can check if a network is legacy network using the following command or the Google Cloud console. The Mode field indicates the type of network.
```
gcloud compute networks list
NAME             MODE    IPV4_RANGE     GATEWAY_IPV4
default          auto
legacy-network   legacy  10.240.0.0/16  10.240.0.1
test-network     custom
```
Required 'compute.instances.create' permission for 'projects/PROJECT_ID/zones/ZONE/instances/test-inst'

If you get this message, the account you are logged in with does not have IAM permissions necessary for creating an instance. Refer to IAM Permissions for detailed information on the roles required for creating instances. You can check if the IAM policy associated with your project grants you any of the following roles: OWNER, EDITOR or compute.instanceAdmin.v1. For creating instances in Shared VPC you would additionally need the compute.networkUser role. In the below example, the account [email protected] doesn’t have sufficient IAM permissions to create an instance. For more detailed instructions see the Granting, Changing, and Revoking Access to Resources IAM guide.
```
gcloud projects get-iam-policy PROJECT_ID
bindings:
- members:
  - user:[email protected]
    role: roles/owner
- members:
  - serviceAccount:[email protected]
  - serviceAccount:[email protected]
    role: roles/editor
- members:
  - user:[email protected]
    role: roles/viewer
etag: BwUjMhXbSPU=
version: 1
```
You can ask your project owner or editor to either grant you a role of OWNER, EDITOR, or compute.instanceAdmin.v1. You would need compute.networkUser role as well if you are associating any interface with a subnet belonging to Shared VPC.
```
gcloud projects set-iam_policy --member user:[email protected] --role roles/editor
```

I can't connect to a secondary interface's internal IP address

Check firewall rules to see that they allow connectivity to the VM secondary interface. You can check by looking at the firewall rules for the network that is attached to the secondary interface by either visiting Google Cloud console and clicking the appropriate VPC network or by using the following gcloud command.
```
gcloud compute firewall-rules list --filter='network:NETWORK_NAME'
```
Check if you are trying to connect to a secondary interface from an Internet address or from outside of the secondary interface's network. You can only connect to an interface's internal IP from within its network. If you need to reach that interface from outside the network, you can assign an external IP address to your secondary interface.
Check if you are trying to connect to the secondary interface's internal IP from outside the subnet the secondary interface is attached to, either from another subnet of the same network or from a peered network. Multiple Network Interfaces per instance explains the interplay between VPC Peering and VM instances with multiple interfaces. In order to reach secondary interfaces from outside the interface's subnet, you might have to configure routes on the VM. See DHCP behavior with multiple network interfaces for details on how DHCP programs default routes in the VM.
If you're trying to access an IPv6 address, also see I can't connect to a secondary interface's IPv6 address.

I can't connect to a secondary interface using an external IP address

The DHCP server programs a default route only on the primary network interface of the VM. If you want to connect to the secondary interface using an external IP, there are two options. If you only need to connect outside the network on the secondary network interface, you can set a default route on that network interface. Otherwise, you can use the approach described in the tutorial, Configure routing for an additional interface, to configure a separate routing table using source-based policy routing in your VM.

If you're trying to access an IPv6 address, also see I can't connect to a secondary interface's IPv6 address.

I can't connect to a secondary interface's IPv6 address

If you're trying to access an IPv6 address, check that the VM has google-guest-agent version 20220603.00 or later installed. For more information about managing google-guest-agent versions, see Guest environment.

If you have an earlier version of the google-guest-agent, the secondary interfaces do not receive an IPv6 subnet route. We recommend that you update the guest agent so that the routes are configured correctly.

However, as a workaround, you can instead create a startup script to make the following configuration change for each secondary interface.

sudo sysctl -w net.ipv6.conf.INTERFACE_NAME.accept_ra_rt_info_max_plen=128

Replace INTERFACE_NAME with the name of the interface—for example, eth1 or ens5.

I am having connectivity issues when using a netmask that is not /32

By default, the instance metadata server only responds to ARP requests for the default gateway.

To configure interfaces with a netmask other than /32, you should create an image using the flag --guest-os-features MULTI_IP_SUBNET and use it to create your instance. For example, if you are using a debian-9 based image, you can create an image using following command:

gcloud compute images create debian-9-multi-ip-subnet \
     --source-disk debian-9-disk \
     --source-disk-zone us-west1-a \
     --guest-os-features MULTI_IP_SUBNET

To view the guest features configured on the image, run the gcloud compute images describe command on the guest image.

gcloud compute images describe debian-9-multi-ip-subnet

For more information about creating custom images, see Create, delete, and deprecate custom images.

I can't resolve the internal hostname of a VM with multiple network interfaces

When a DNS query is executed with an internal hostname, it resolves to the primary network interface (nic0) of the instance.

If nic0 is attached to a VPC network that is different from the VPC network of the instance issuing the DNS query, then the DNS query fails. Internal DNS is resolved only within a given VPC network. For more information, see DNS resolution with multiple network interfaces.

Troubleshooting using the serial console

It is often a good idea to enable the serial console on a VM to debug configuration-related issues. You can enable the serial console for interactive debugging by following the steps in Troubleshooting using the serial console.