Using Terraform to Build a Shared VPC Network in GCP

Photo by Leo_Visions on Unsplash

As always if this article saved you precious time, please be sure to applaud and share it. If there is something you would like to add to it please be sure to comment on it. This article was not written by a GPT but a human being so feel free to reach out!

I wrote an article a little while back that walked through the process of setting up a Cloud Composer environment in a Shared VPC network. As a data engineer who sometimes wears an infrastructure engineer hat, my intention with that post was to show at the console level what a network or infrastructure engineer would need to do to set up a Shared VPC network, a subnet, and secondary IP ranges, appropriate IAM service accounts and roles needed for an enterprise-ready Cloud Composer environment. The second half of that article shows how a data infrastructure engineer would use Terraform to create the Cloud Composer environment and properly reference the Shared VPC Network in the code. Now, after seeing how much traffic that article has gotten, I would like to write its companion so-to-speak. I hope this is short, sweet and painless. I will show one way to go about the process of setting up a Shared VPC completely through IaC using Terraform. If you are here hopefully you understood what is happening in my previous article and been tasked to set up the networking bits via IaC or that other article is foreign to you and you just want to get to the business of c̶o̶p̶y̶i̶n̶g̶ comparing my Terraform code to what you have and pray that it gets you unstuck. Either way, welcome!

I am not going to hit you with all the reasons why IaC is better than doing things at the console. I am sure you already know that. I am making the assumption that because you are trying to set up some non-trivial networking infrastructure I am not going to insult your intelligence by telling you that you need a GCP project for your shared VPC host project, the proper IAM service accounts, roles, etc. You should have the basics down before reading further. I generally suggest running through your infrastructure setup once via the console and/or CLI in order to understand how the pieces fit together. This makes writing your Terraform code much easier. If you do have questions about the difference between a host project, a service project, or why we need to define secondary IP ranges, I will point you to my first article.

I am a big fan of Terraform modules because I don’t like reinventing the wheel and I like to be efficient. For this little project I am going to be using the Terraform network module to create the VPC network with the appropriate subnet and secondary IP addresses and the shared VPC network module.

Without further adieu let’s hop right into the code.

# main.tf

module "vpc" {
  source  = "terraform-google-modules/network/google"
  version = "~> 9.0"

  project_id      = var.project_id
  network_name    = "shared-vpc"
  # Choose GLOBAL or REGIONAL based on the geographical distribution of your GCP resources.
  routing_mode    = "GLOBAL"
  # Makes this project a shared VPC host.
  shared_vpc_host = true

  # I am matching the subnet and secondary ranges for setting up a Cloud Composer Environment in a shared vpc.
  subnets = [
    {
      subnet_name           = "composer-gke-node-subnet"
      subnet_ip             = "10.0.1.0/24"
      subnet_region         = "us-central1"
      subnet_private_access = "true"
      description           = "Subnet to hold the GKE nodes for Cloud Composer cluster."
    }
  ]

  secondary_ranges = {
    composer-gke-node-subnet = [
      {
        range_name    = "composer-control-plane"
        ip_cidr_range = "10.5.0.0/16"
      },
      {
        range_name    = "composer-gke-svc"
        ip_cidr_range = "10.0.2.0/24"
      },
      {
        range_name    = "composer-pods"
        ip_cidr_range = "10.0.4.0/23"
      }
    ]
  }

  ingress_rules = [
    {
      name        = "shared-vpc-composer-allow"
      description = "Allow rules for Cloud Composer"
      source_ranges = [
        "10.0.1.0/24",
        "10.0.2.0/24",
        "10.0.4.0/23"
      ]
      allow = [
        {
          protocol = "all"
        }
      ]
    }
  ]

}

# variables.tf

variable "host_project_id" {
  type        = string
  description = "GCP project ID of the Host Project."
}

Keep in mind that you can parameterize your maint.tf to your heart’s content. I just kept this example as simple as possible while showing the most important parts.

Once you have this set up run the following commands:

$ terraform init
$ terraform fmt
$ terraform validate
$ terraform plan
$ terraform apply

After you have run your Terraform plan make sure everything looks good from the console. Navigate to the VPC Networks GCP console page, click into the VPC Networks tab and you should see your new VPC. You can click on the VPC you just created and view your new subnet. Click on your subnet name and view details to verify your CIDR ranges are all there and at the correct value. This is what it should look like.

Subnets Detail Page in GCP Console

Under the Firewall Rules tab verify that your rules were applied

Firewall Rules details Page from GCP Console

Finally, make sure that the VPC and subnet were shared with the host project that was specified in the terraform code. From the Shared VPC tab select your host project. You should now see your subset in the Shared VPC Network.

Shared VPC Page from GCP Console

Just like that you are now managing your Shared VPC that will host your Cloud Composer environment!

If you are looking for more help modernizing your tech stack with GCP we at Real Kinetic are here to help. Reach out and one of our amazing engineers will get in touch with you.