Building Kubernetes cluster on AWS cloud using Terraform and Ansible

Let’s learn how to automate the Kubernetes cluster building on the AWS cloud using Terraform and Ansible. Terraform is the infrastructure provisioning tool, whereas Ansible is the configuration management and deployment tool. Before getting started with the hands-on, you need to have the following requirements.

• An account on AWS cloud
• AWS IAM user with administrator access credentials
• Amazon EC2 key pair named ec2-user
• Linux Ubuntu 18.04 LTS operating system is preferred

For better understanding, the steps are broken down into two major sections. It will help you effectively understand the automated building of the Kubernetes cluster on the AWS cloud using Terraform and Ansible.

1. Terraform-managed AWS cloud infrastructure provisioning for Kubernetes cluster
2. Ansible-managed Kubernetes cluster deployment on Amazon EC2 instances

Terraform-managed AWS cloud infrastructure provisioning for Kubernetes cluster

Configure AWS IAM user with administrator access credentials

Here, we configure an AWS IAM user profile named tuto using the AWS CLI. Please don’t change the AWS IAM user profile name because Terraform inherits this credentials while provisioning infrastructure.

# Configure CLI for AWS IAM user.
$ aws configure --profile tuto

Install Terraform on Linux Ubuntu 18.04 LTS operating system

Terraform is an Open Source infrastructure provisioning tool/software that allows you to provision and manage infrastructure components like servers, containers, storage, network, etc. Please use the below steps to download, extract, and install Terraform.

# Download Terraform.
$ wget https://releases.hashicorp.com/terraform/0.14.4/terraform_0.14.4_linux_amd64.zip

# Extract Terraform from archive.
$ unzip terraform_0.14.4_linux_amd64.zip

# Install Terraform.
$ sudo mv terraform /usr/local/bin

# Check Terraform version.
$ terraform --version

Download Kubernetes starter-kit from SloopStash GitHub account

Let’s download the Kubernetes starter-kit repository from SloopStash GitHub account to the local filesystem path. The Kubernetes starter-kit repository contains the Terraform configurations, Ansible playbooks, and Ansible roles that are required for automated building of the Kubernetes cluster on the AWS cloud.

# Download Kubernetes starter-kit from GitHub to local filesystem path.
$ wget https://github.com/sloopstash/kickstart-kubernetes/archive/v1.2.1.zip

# Extract Kubernetes starter-kit from archive.
$ unzip v1.2.1.zip -d /opt/kickstart-kubernetes

# Change ownership of Kubernetes starter-kit directory.
$ sudo chown -R $USER:$USER /opt/kickstart-kubernetes

Provision AWS resources for Kubernetes cluster using Terraform

Here, we generate a plan from the Terraform configuration and apply the same to provision resources on AWS IAM, Amazon VPC, and Amazon EC2 that are required for running the Kubernetes cluster. After provisioning AWS resources, Terraform generates an output that contains the public IPs of provisioned Amazon EC2 instances, save them somewhere for future use.

# Switch to Kubernetes starter-kit directory.
$ cd /opt/kickstart-kubernetes/terraform

# Initialize Terraform configuration.
$ terraform init

# Store variables of Terraform configuration as environment variables.
$ export TF_VAR_env=STG
$ export TF_VAR_stg_vpc_cidr_blk=10.2.0.0/16
$ export TF_VAR_stg_ec2_key_pair=ec2-user

# Generate plan using Terraform configuration.
$ terraform plan -out stg-tfm-base-cfg.tfplan

# Apply plan generated by Terraform configuration.
$ terraform apply stg-tfm-base-cfg.tfplan

Ansible-managed Kubernetes cluster deployment on Amazon EC2 instances

Install Ansible on Linux Ubuntu 18.04 LTS operating system

Ansible is an Open Source configuration management and software deployment tool. Please use the below steps to install Ansible.

# Install required system packages.
$ sudo apt install python-pip

# Install Ansible package from repository.
$ sudo pip install ansible==2.8.2

# Check Ansible version.
$ ansible --version

Deploy Kubernetes on Amazon EC2 instances using Ansible

Let’s execute the Ansible playbook to deploy Docker and Kubernetes on Amazon EC2 instances to form Kubernetes cluster. Here, Amazon EC2 instances are the Ansible nodes.

# Switch to Kubernetes starter-kit directory.
$ cd /opt/kickstart-kubernetes/ansible

# Update hosts in Ansible inventory with public IPs of Amazon EC2 instances.
$ vim inventory/stg/hosts

# Copy Amazon EC2 SSH private key to authenticate SSH server running on Ansible nodes.
$ cp ec2-user.pem secret/node.pem

# Trigger first time deployment on Kubernetes Ansible nodes.
$ ansible-playbook playbook/kubernetes.yml -i inventory/stg --tags "setup, configure, start"

So far, we have provisioned resources required for running Kubernetes cluster and deployed Kubernetes with Docker on Amazon EC2 instances. Now you can SSH into master Kubernetes node to manage the cluster.

# SSH to master Kubernetes node.
$ ssh ec2-user@<MASTER_K8S_NODE_IP>

# List Kubernetes nodes.
$ kubectl get nodes -o wide

Hope it helps. Thank you.