Mini-Project: Simple Automation Using Ansible

 In this Mini-Project, I implemented configuration management technology using an Ansible and running a simple service, Apache. And I’ll install it on 2 slaves at once. So, what are they?

Configuration Management

Configuration management systems are designed to make controlling large numbers of servers easy for administrators and operations teams. They allow controlling many different systems in an automated way from one central location.

Ansible

Ansible is an open-source software provisioning, configuration management, and application-deployment tool. It includes its own declarative language to describe system configuration.

Ansible is the simplest solution for configuration management available. It’s designed to be minimal in nature, consistent, secure and highly reliable, with an extremely low learning curve for administrators, developers, and IT managers.

How Does Ansible Work?

Ansible works by configuring client machines from a computer that has the Ansible components installed and configured.

It communicates over normal SSH channels to retrieve information from remote machines, issue commands, and copy files. Because of this, an Ansible system does not require any additional software to be installed on the client computers.

This is one way that Ansible simplifies the administration of servers. Any server that has an SSH port exposed can be brought under Ansible’s configuration umbrella, regardless of what stage it is at in its life cycle. This means that any computer that you can administer through SSH, you can also administer through Ansible.

Configuration files are mainly written in the YAML data serialization format due to its expressive nature and its similarity to popular markup languages. Ansible can interact with hosts either through command-line tools or its configuration scripts, which are known as Playbooks.

Apache

Apache web server is a popular open-source HTTP web server tool that is widely used for the deployment of webpages. It can be installed in any operating system.

Step 1 — Installing Ansible

To begin using Ansible as a means of managing your various servers, you need to install the Ansible software on at least one machine as a master.

To get the latest version of Ansible for Ubuntu, you can add the project’s PPA (personal package archive) to your system. Before doing this, you should first ensure that you have the package installed.

- Update package

# apt update

- Install a few prerequisite packages

# apt install software-properties-common

- Add the Ansible PPA (personal package archive)

# apt-add-repository ppa:ansible/ansible

- Update the package with the newly added repo

# apt update

- Install Ansible

# apt install ansible

Step 2 — Configuring Ansible Hosts

- Access key

As mentioned previously, Ansible primarily communicates with client computers through SSH. While it certainly has the ability to handle password-based SSH authentication, using SSH keys can help to keep things simple. But, the slaves here using EC2 Instances, so we use a keypair that has applied on to connect over SSH.

- Install a prerequisite packages

# apt update
# apt install python

Because Ansible uses a python interpreter to run its modules, we need to install Python 2 on the host in order for Ansible to communicate with it. Repeat this process for each server you intend to control with your Ansible server.

- Adding client

# nano /etc/ansible/hosts

Ansible keeps track of all of the servers that it knows about through a hosts file. We need to set up this file first before we can begin to communicate with our other computers.

And then add the following line above. It means that we have 2 groups named master that is installed Ansible and slave that have no services. There are vars that mean ubuntu as a user which will be connected by the master because the direct connection as a root user is not allowed. And don't forget the access key, I used the same key on all machine to make it easier.

- Check connection

# ansible -m ping all

Now that we have our hosts set up and enough configuration details to allow us to successfully connect to our hosts, we can try out our very first command.

And done! the slaves have been connected with the master now.

Step 3 — Make an Automation

- Make a directory

# mkdir /ansible
# cd /ansible

This directory is used to store some files for starting new automation. And make sure we already on that directory after it.

- Create a sample web page

# nano index.html

Fill in web content as needed and then name the file with index.html. But I’ll fill it with:

<!DOCTYPE html>
<html>
<body>  <h1>IT WORKS!</h1>
  <h2>Now you’re running webserver from simple automation</h2></body>
</html>

- Create a Playbooks

Playbooks are YAML files containing a series of directives to automate the provisioning of a server.

YAML relies on indentation to serialize data structures. For that reason, when writing playbooks and especially when copying examples, you need to be extra careful to maintain the correct indentation.

# nano playbook.yml

The name for the playbooks file does not need to be considered, the most important is the format of the playbooks file, which is .yaml or .yml.

---
- name: Install and run apache
  hosts: slave
  become: true
  become_user: root
  gather_facts: true
  tasks:
    
    - name: Install service
      yum:
        name: apache2
        state: latest    - name: Create content
      template:
        src: index.html
        dest: /var/www/html/index.html    - name: Run service
      service:
        name: apache2
        state: started

And here I’ll create simple playbooks for our 2 slaves that containing the command for changing user to root because we gonna install a package which is Apache2, copying the web page that we have been made to the directory of web server, and then restarting the Apache2.

- Let's automate!

# ansible-playbook playbook.yml

It took a few minutes to run those tasks on the slaves.

Step 3 — See the Result

- Check the installations

# service apache2 status

- Check the Apache2 service

we are going to create a VPC using Ansible in AWS.

 > cd /etc/ansible/roles

> mkdir vpc-roles

> cd vpc-roles 

> mkdir defaults files vars templates test tasks 

> cd tasks 

> vim main.yml 

---

 - name: create VPC

   ec2_vpc_net:

     name: "{{ vpc_name }}"

     cidr_block: "{{ vpc_cidr }}"

     region: "{{ region }}"

     state: present

     aws_access_key: "{{ aws_access_key }}"

     aws_secret_key: "{{ aws_secret_key }}"

   register: vpc

 - name: Set VPC ID in variable

   set_fact:

     vpc_id: "{{ vpc.vpc.id }}"

 - name: associate subnet to the VPC

   ec2_vpc_subnet:

    state: present

    vpc_id: "{{ vpc_id }}"

    region: "{{ region }}"

    cidr: "{{ subnet_cidr }}"

    aws_access_key: "{{ aws_access_key }}"

    aws_secret_key: "{{ aws_secret_key }}"

    map_public: yes

    resource_tags:

      Name: "{{ subnet_name }}"

   register: subnet

- name: create IGW

   ec2_vpc_igw:

    vpc_id: "{{ vpc_id }}"

    region: "{{ region }}"

    aws_access_key: "{{ aws_access_key }}"

    aws_secret_key: "{{ aws_secret_key }}"

    state: "present"

    tags:

      Name: "{{ igw_name }}"

   register: igw

 - name: Route IGW

   ec2_vpc_route_table:

    vpc_id: "{{ vpc_id }}"

    region: "{{ region }}"

    aws_access_key: "{{ aws_access_key }}"

    aws_secret_key: "{{ aws_secret_key }}"

    subnets:

      - "{{ subnet.subnet.id }}"

    routes:

      - dest: 0.0.0.0/0

        gateway_id: "{{ igw.gateway_id  }}"

    tags:

      Name: "{{ route_name }}"

- name: Create Security Group

   ec2_group:

    name: Web DMZ

    description: DMZ Security Group

    vpc_id: "{{ vpc_id }}"

    region: "{{ region }}"

    aws_access_key: "{{ aws_access_key }}"

    aws_secret_key: "{{ aws_secret_key }}"

    rules:

      - proto: tcp

        ports:

        - 80

        cidr_ip: 0.0.0.0/0

      - proto: tcp

        ports:

        - 22

        cidr_ip: 0.0.0.0/0

   register: security_group

 - name: create a new ec2 key pair, returns generated private key

   ec2_key:

    aws_access_key: "{{ aws_access_key }}"

    aws_secret_key: "{{ aws_secret_key }}"

    name: "{{ keypair_name }}"

    region: "{{ region }}"

   register: keypair

   tags:

    - ec2_key

 - debug:

      var: keypair

   tags:

    - ec2_key

- name: Copy EC2 Key Pair private key locally

   copy: content="{{ keypair.key.private_key }}" dest={{ ec2_key_directory }}key.ppk

   when: keypair.changed == true

   tags:

    - ec2_key

 - name: Create EC2 Instance

   ec2:

    image: ami-087c17d1fe0178315

    wait: yes

    instance_type: t2.small

    region: "{{ region }}"

    group_id: "{{ security_group.group_id }}"

    vpc_subnet_id: "{{ subnet.subnet.id }}"

    key_name: "{{ keypair.key.name  }}"

    count_tag: "{{ ec2_tag }}"

    exact_count: 1

    aws_access_key: "{{ aws_access_key }}"

    aws_secret_key: "{{ aws_secret_key }}"

  ______________________________________________________________

> cd vars 

> vim main.yml

---

  

hosts: all

vars:

 ansible_python_interpreter: /usr/bin/python3

aws_access_key: AA2YKZL3BBPLMX7X2F

aws_secret_key: :wqXl9LJ7KRtyR05fgdSOV/GeQFsViCzM0ClwQPMiP

region: us-east-1

# VPC

vpc_cidr: 10.10.0.0/24

vpc_name: "Ansible VPC3"

# Subnet

subnet_name: "Ansible Subnet"

subnet_cidr: 10.10.0.0/26

igw_name: "Traffic IGW"

#route table name

route_name: "Public Ansible"

securitygroup_name: "Ansible Security Group"

ec2_tag: "WebServer"

#key storage location

ec2_key_directory: "/home/abidmoon/"

____________________________________

> cd tests 

> vim tests.yml

---

- hosts: localhost

  remote_user: root

  roles:

    - vpc-roles

__________________________

ansible-playbook test.yml

done 

Ansible Project 1

 In this tutorial, we are going to create a VPC using Ansible in AWS. Our VPC will include a router, subnet, IGW, Security Group and definitely an EC2 instance.

This is the topology we would create. (I have mentioned the Ansible Module so as to make it easier)

Prerequisites

• IAM User with the required permission.
• Boto installed. (Ansible uses the Boto Python library to handle AWS management)

If you are planning to improve your IAM security, please read my guide on it.

1. Creating a custom Role

We would first create an AWS VPC Role.

1	ansible-galaxy init aws-vpc

We would be using the two files below from our aws-vpc role.

1 2 3 4

- tasks    - main.yml - vars    -main.yml

Variables

We will place the variables in the vars/main.yml file. This allows the role to be easily reusable. Modify the variables depending on your requirement but replace your AWS Access and Secret key.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

--- #vars/main.yml   aws_access_key: "" aws_secret_key: "" region: "us-east-1"   # VPC vpc_cidr: 10.10.0.0/24 vpc_name: "Ansible VPC"   # Subnet subnet_name: "Ansible Subnet" subnet_cidr: 10.10.0.0/26   # Internet Gateway Name igw_name: "Traffic IGW"   securitygroup_name: "Ansible Security Group"   ec2_tag: "WebServer"   #The local path to which we would save our EC2 Private Key ec2_key_directory: "/home/ansible/roles/aws-vpc/" keypair_name: "ec2_key_pair"

Do not store your AWS credentials as a variable! Follow the best practice

2. Creating the VPC

Our VPC will include the following components.

• VPC
• Subnet
• Router
• IGW (Internet Gateway)
• Security Group
• EC2 Instance

We will write all the code below in the tasks/main.yml file.

a) VPC

To create a VPC, we have to use the ec2_vpc_net module. We will register the module returned data in the “vpc” variable.

1 2 3 4 5 6 7 8 9

- name: create VPC   ec2_vpc_net:     name: "{{ vpc_name }}"     cidr_block: "{{ vpc_cidr }}"     region: "{{ region }}"     state: present     aws_access_key: "{{ aws_access_key }}"     aws_secret_key: "{{ aws_secret_key }}"   register: vpc

b) Creating a subnet and associating it with an Internet Gateway

Now that we have successfully created a VPC. We would need to create a subnet and an IGW. We need an IGW so that our public subnet can access the internet.

In our subnet, we have set map_public to “YES”. This means instances created in this subnet is associated with a public IP address.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

- name: associate subnet to the VPC   ec2_vpc_subnet:    state: present    vpc_id: "{{ vpc_id }}"    region: "{{ region }}"    cidr: "{{ subnet_cidr }}"    aws_access_key: "{{ aws_access_key }}"    aws_secret_key: "{{ aws_secret_key }}"    map_public: yes    resource_tags:      Name: "{{ subnet_name }}"   register: subnet   - name: create IGW   ec2_vpc_igw:    vpc_id: "{{ vpc_id }}"    region: "{{ region }}"    aws_access_key: "{{ aws_access_key }}"    aws_secret_key: "{{ aws_secret_key }}"    state: "present"    tags:      Name: "{{ igw_name }}"   register: igw

c) Routing traffic to IGW and creating a security group

Now, we would need to route the traffic to the IGW via the route table and create a security group.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

- name: Route IGW    ec2_vpc_route_table:     vpc_id: "{{ vpc_id }}"     region: "{{ region }}"     aws_access_key: "{{ aws_access_key }}"     aws_secret_key: "{{ aws_secret_key }}"     subnets:       - "{{ subnet.subnet.id }}"     routes:       - dest: 0.0.0.0/0         gateway_id: "{{ igw.gateway_id  }}"     tags:       Name: "{{ route_name }}"   # update the CIDR address in the SSH port section.    - name: Create Security Group    ec2_group:     name: Web DMZ     description: DMZ Security Group     vpc_id: "{{ vpc_id }}"     region: "{{ region }}"     aws_access_key: "{{ aws_access_key }}"     aws_secret_key: "{{ aws_secret_key }}"     rules:       - proto: tcp         ports:         - 80         cidr_ip: 0.0.0.0/0       - proto: tcp         ports:         - 22         cidr_ip: 0.0.0.0/0    register: security_group

d) Creating an EC2 Key Pair and an EC2 instance

This is our last step, we will create an EC2 Key Pair and an EC2 instance.

Key points:

• if you already have an EC2 key pair, skip the EC2 key pair and update the key_name with the name of the key pair.
• exact_count is important! It determines the number of instances is created.
• Image ID can be found on AWS Marketplace.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

- name: create a new ec2 key pair    ec2_key:     aws_access_key: "{{ aws_access_key }}"     aws_secret_key: "{{ aws_secret_key }}"     name: ec2_keypair     region: "{{ region }}"    register: keypair   - name: Copy EC2 Private Key locally so it can be later on used to SSH into the instance   copy: content="{{ keypair.key.private_key }}" dest={{ ec2_key_directory }}key.ppk   when: keypair.changed == true    - name: Create EC2 server    ec2:     image: ami-467ca739     wait: yes     instance_type: t2.micro     region: "{{ region }}"     group_id: "{{ security_group.group_id }}"     vpc_subnet_id: "{{ subnet.subnet.id }}"     key_name: "{{ keypair.key.name  }}"     count_tag:       Name: apacheserver     exact_count: 1     aws_access_key: "{{ aws_access_key }}"     aws_secret_key: "{{ aws_secret_key }}"

This creates a simple VPC but creating a complex VPC is still achievable with Ansible, as it provides all the necessary modules for a VPC creation in the AWS.