2.4 Working with Terraform Modules: How to Organize and Reuse Infrastructure Code

Kubernetes Infrastructure Made Easy: Level Up with Terraform Modules

Kubernetes is powerful, but managing the infrastructure that supports it can be complex. Terraform helps you define and manage that infrastructure as code. But as your Kubernetes environment grows, your Terraform code can become unwieldy. That's where Terraform Modules come in – think of them as pre-packaged, reusable building blocks for your infrastructure!

What are Terraform Modules? (Think Legos for Infrastructure!)

Imagine you're building a Lego castle. You could start from scratch with individual Lego bricks. Or, you could use pre-built Lego sections, like a tower or a gate, that you can easily attach to your main castle.

That's what Terraform Modules are! They are self-contained packages of Terraform code that define a specific piece of infrastructure, like a Kubernetes cluster, a load balancer, or a database. You can reuse these modules across different environments (development, staging, production) or even different projects, saving you time and effort.

Why Use Terraform Modules for Kubernetes?

• Reusability: Avoid duplicating code. Define a Kubernetes cluster once and reuse it for multiple environments.

• Organization: Break down complex infrastructure into manageable chunks.

• Consistency: Ensure consistent configurations across environments.

• Abstraction: Hide the complexity of the underlying infrastructure from users. They just need to use the module, not understand every detail.

• Maintainability: Changes to a module automatically propagate to all instances of that module.

A Real-World Example: Creating a Kubernetes Cluster with a Module

Let's say you want to create a Kubernetes cluster on AWS using Terraform. Instead of writing all the code from scratch, you can use a pre-built module.

Here's a simplified example:

1. Create a directory structure:

├── modules/
│   └── kubernetes-cluster/
│       ├── main.tf       # Define the Kubernetes cluster resources
│       ├── variables.tf  # Define input variables (e.g., cluster name, region)
│       └── outputs.tf    # Define outputs (e.g., kubeconfig)
├── main.tf             # Root module to use the Kubernetes cluster module
└── variables.tf        # Define input variables for the root module

2. modules/kubernetes-cluster/main.tf (Simplified):

resource "aws_eks_cluster" "this" {
  name     = var.cluster_name
  role_arn = var.cluster_role_arn

  vpc_config {
    subnet_ids = var.subnet_ids
  }

  version = "1.28"
}

resource "aws_eks_node_group" "this" {
  cluster_name    = aws_eks_cluster.this.name
  node_group_name = "${var.cluster_name}-nodes"
  node_role_arn   = var.node_role_arn
  subnet_ids      = var.subnet_ids

  scaling_config {
    desired_size = var.node_desired_size
    max_size     = var.node_max_size
    min_size     = var.node_min_size
  }

  version = "1.28"
}

3. modules/kubernetes-cluster/variables.tf:

variable "cluster_name" {
  type = string
  description = "The name of the Kubernetes cluster"
}

variable "cluster_role_arn" {
  type = string
  description = "ARN of the IAM role for the Kubernetes cluster"
}

variable "node_role_arn" {
  type = string
  description = "ARN of the IAM role for the worker nodes"
}

variable "subnet_ids" {
  type = list(string)
  description = "List of subnet IDs for the Kubernetes cluster"
}

variable "node_desired_size" {
  type = number
  default = 1
  description = "Desired number of nodes"
}

variable "node_max_size" {
  type = number
  default = 3
  description = "Maximum number of nodes"
}

variable "node_min_size" {
  type = number
  default = 1
  description = "Minimum number of nodes"
}

4. modules/kubernetes-cluster/outputs.tf:

output "cluster_name" {
  value       = aws_eks_cluster.this.name
  description = "The name of the Kubernetes cluster"
}

5. main.tf (Using the module):

module "my_eks_cluster" {
  source = "./modules/kubernetes-cluster"

  cluster_name   = "my-dev-cluster"
  cluster_role_arn = var.cluster_role_arn
  node_role_arn = var.node_role_arn
  subnet_ids      = var.subnet_ids
  node_desired_size = 2
  node_max_size = 4
  node_min_size = 2
}

output "cluster_name" {
  value = module.my_eks_cluster.cluster_name
}

6. variables.tf:

variable "cluster_role_arn" {
  type = string
  description = "ARN of the IAM role for the Kubernetes cluster"
}

variable "node_role_arn" {
  type = string
  description = "ARN of the IAM role for the worker nodes"
}

variable "subnet_ids" {
  type = list(string)
  description = "List of subnet IDs for the Kubernetes cluster"
}

Explanation:

• The modules/kubernetes-cluster directory contains all the code needed to create a Kubernetes cluster.

• The main.tf file within the module defines the necessary AWS resources.

• The variables.tf file defines the input variables for the module, such as the cluster name, role ARNs, and subnet IDs.

• The outputs.tf file defines the output variables, such as the cluster name and endpoint.

• In the root main.tf, the module "my_eks_cluster" block uses the module defined in the modules/kubernetes-cluster directory. You provide values for the module's input variables.

Now, running terraform init, terraform plan, and terraform apply will create a Kubernetes cluster based on the module's configuration. You can reuse this module to create other clusters with different configurations, just by changing the variable values.

Architectural Diagram:

+-----------------------+     +-----------------------+     +-----------------------+
|       Root Module     |     |   Module Definition   |     |   AWS Infrastructure  |
|   (main.tf, etc.)   |     | (modules/...)        |     |    (Kubernetes, etc.)|
+-----------------------+     +-----------------------+     +-----------------------+
| Uses 'kubernetes-    | --> |  Defines resources,   | --> |   Creates/Updates/   |
| cluster' module      |     |  variables, outputs   |     |   Deletes resources    |
| Provides input       |     |                       |     |                       |
| variables            |     |                       |     |                       |
+-----------------------+     +-----------------------+     +-----------------------+

Challenge: Managing Module Versions

One common challenge is managing different versions of your modules. What if you update a module, and you don't want all your environments to automatically use the latest version?

Solution: Versioning with Git and Terraform Registry

• Git Tags: Tag your module code in Git with semantic versions (e.g., v1.0.0, v1.1.0).

• Terraform Registry: Package and publish your modules to the Terraform Registry. This allows you to specify versions in your source attribute:

module "my_eks_cluster" {
  source  = "your-org/kubernetes-cluster/aws"
  version = "1.0.0"

  # ... (rest of your configuration) ...
}

This way, you have precise control over which versions of your modules are used in each environment. You can upgrade your modules gradually, testing changes in non-production environments first.

Conclusion

Terraform Modules are essential for managing Kubernetes infrastructure effectively. By organizing and reusing your infrastructure code, you can save time, reduce errors, and create a more consistent and manageable environment. So, start building your own Lego sets (modules) and level up your Kubernetes infrastructure game!