Keeping your Staging RDS up2date with terraform

I think I do not need to explain why terraform is a flexible and pretty straight forward tool (once you get to know it a little).

What I Love about it is mainly the time to market and the fact I through away a bunch of bash + Ansible scripts which had a lot of “logic” and facts which were replaced by ~100 lines of “terracode” / HCL …and under 2 hours in total - I got the job done !

A Simplified setup …

To emphasize what i’m trying to achieve here see the following diagram:

Imgur

Greatest and latest

Getting the latest RDS snapshot id was easy as:


# RDS common name
variable "prod_rds_common_name" {
  default = "tikal"
}

# Get latest snapshot from production DB
data "aws_db_snapshot" "latest_db_snapshot" {
    most_recent = true
    db_instance_identifier = "${var.prod_rds_common_name}"
}

The above snippet basically fetches the latest snapshot_id which you can get with the following aws-cli command: aws rds describe-db-snapshots.

The fun fact, using terraform I don’t need to update the snapshot ID each time I want to get the greatest and latest DB state, So potentially if I tell my Jenkins / CI server to do this periodically I can provide my team with an up2date version, sanitize the data (keeping things safe …) and attach it to the staging tenant.

Attaching my new RDS to Staging …

Today our nodejs applications were configured to use a rds-endpoint which looks like tikal.ctemagubppe2.eu-west-1.rds.amazonaws.com

So the next issue had was:

Configuring Staging servers to use the latest RDS

At first we discussed having Jenkins run an Ansible playbook straight after Terraform and using terraform’s outputs something like:


output "rds_address" {
  value = "${aws_db_instance.db-stg.address}"
}

And then execute ansible with

ansible-playbook stg-app.yml -e rds_db_url=`terraform output rds_address`

Considering our node.js app config uses something like:


"db": {
    "host": "tikal.ctemagubppe2.eu-west-1.rds.amazonaws.com",
    "name": "<prod_db_name>",
    "user": "<username>",
    "password": "<passowrd>"
  },

BUT Luckily, Terraform helps out quite easily with it’s great aws api integrations … A simple change to my configuration making it something like:

"db": {
    "host":  "stg-db.dev.tikal.io",
    "name": "<prod_db_name>",
    "user": "<username>",
    "password": "<passowrd>"
  },

Which means I do not need to configure my monolith's anymore … because:

First, I fetch the RDS ip address (considering it’s in a private subnet …)

data "dns_a_record_set" "db-stg" {
  host = "${aws_db_instance.db_stg.address}"
}

Then create/update our private route53 zone address with the new ip_address

data "aws_route53_zone" "dev-primary" {
    name = "dev.tikal.io."
}
resource "aws_route53_record" "db-stg" {
  zone_id = "${data.aws_route53_zone.dev-primary.zone_id}"
  name    = "db-stg.dev.tikal.io"
  type    = "A"
  ttl     = "300"
  records = ["${data.dns_a_record_set.db-stg.addrs}"]
}

And sit back for ~10/15 minuets and your good to go !

The full code looks like this:

# variables.tf ...
# Provider
variable "profile" {
  description = "your default / custom aws profile see your ~/.aws/credentials file for details"
  default     = "tikal"
}
variable "region" {
  description = "your default aws region"
  default     = "eu-west-1"
}
# RDS
variable "prod_rds_common_name" {
  default = "tikal"
}
# main.tf
provider "aws" {
  region  = "${var.region}"
  profile = "${var.profile}"
}
data "aws_vpc" "prod" {
  tags {
    Name = "tikal-default"
  }
}
data "aws_subnet_ids" "in_main_vpc" {
  vpc_id = "${data.aws_vpc.prod.id}"
}
# Get latest snapshot from production DB
data "aws_db_snapshot" "latest_db_snapshot" {
    most_recent = true
    db_instance_identifier = "${var.prod_rds_common_name}"
}
data "aws_security_group" "db-access" {
  filter {
    name   = "group-name"
    values = ["db-access"]
  }
  filter {
    name   = "vpc-id"
    values = ["${data.aws_vpc.prod.id}"]
  }
}
# Create new staging DB
resource "aws_db_instance" "db_stg" {
  instance_class       = "db.t2.large"
  identifier           = "db-stg"
  availability_zone = "${var.region}b"
  snapshot_identifier  = "${data.aws_db_snapshot.latest_db_snapshot.id}"
  vpc_security_group_ids = ["${data.aws_security_group.db-access.id}"]
  skip_final_snapshot = true
}
data "dns_a_record_set" "db-stg" {
  host = "${aws_db_instance.db_stg.address}"
}
data "aws_route53_zone" "dev-primary" {
    name = "dev.tikal.io."
}
resource "aws_route53_record" "db-stg" {
  zone_id = "${data.aws_route53_zone.dev-primary.zone_id}"
  name    = "db-stg.tikal.io"
  type    = "A"
  ttl     = "300"
  records = ["${data.dns_a_record_set.db-stg.addrs}"]
}
# outputs.tf
output "rds_addrs" {
  value = "${data.dns_a_record_set.db-stg.addrs}"
}
output "aws_db_snapshot_id" {
  value = "${data.aws_db_snapshot.latest_db_snapshot.id}"
}
output "vpc_id" {
    value = "${data.aws_vpc.prod.id}"
}

Hope you enjoyed reading this.

Until next time.

HP

Thank you for your interest!

We will contact you as soon as possible.

Want to Know More?

Oops, something went wrong
Please try again or contact us by email at info@tikalk.com
Thank you for your interest!

We will contact you as soon as possible.

Let's talk

Oops, something went wrong
Please try again or contact us by email at info@tikalk.com