Keeping your Staging RDS up2date with terraform

I think I do not need to explain why terraform is a flexible and pretty straight forward tool (once you get to know it a little).

What I Love about it is mainly the time to market and the fact I through away a bunch of bash + Ansible scripts which had a lot of “logic” and facts which were replaced by ~100 lines of “terracode” / HCL …and under 2 hours in total - I got the job done !

A Simplified setup …

To emphasize what i’m trying to achieve here see the following diagram:

Imgur

Greatest and latest

Getting the latest RDS snapshot id was easy as:


# RDS common name
variable "prod_rds_common_name" {
  default = "tikal"
}

# Get latest snapshot from production DB
data "aws_db_snapshot" "latest_db_snapshot" {
    most_recent = true
    db_instance_identifier = "${var.prod_rds_common_name}"
}

The above snippet basically fetches the latest snapshot_id which you can get with the following aws-cli command: aws rds describe-db-snapshots.

The fun fact, using terraform I don’t need to update the snapshot ID each time I want to get the greatest and latest DB state, So potentially if I tell my Jenkins / CI server to do this periodically I can provide my team with an up2date version, sanitize the data (keeping things safe …) and attach it to the staging tenant.

Attaching my new RDS to Staging …

Today our nodejs applications were configured to use a rds-endpoint which looks like tikal.ctemagubppe2.eu-west-1.rds.amazonaws.com

So the next issue had was:

Configuring Staging servers to use the latest RDS

At first we discussed having Jenkins run an Ansible playbook straight after Terraform and using terraform’s outputs something like:


output "rds_address" {
  value = "${aws_db_instance.db-stg.address}"
}

And then execute ansible with

ansible-playbook stg-app.yml -e rds_db_url=`terraform output rds_address`

Considering our node.js app config uses something like:


"db": {
    "host": "tikal.ctemagubppe2.eu-west-1.rds.amazonaws.com",
    "name": "<prod_db_name>",
    "user": "<username>",
    "password": "<passowrd>"
  },

BUT Luckily, Terraform helps out quite easily with it’s great aws api integrations … A simple change to my configuration making it something like:

"db": {
    "host":  "stg-db.dev.tikal.io",
    "name": "<prod_db_name>",
    "user": "<username>",
    "password": "<passowrd>"
  },

Which means I do not need to configure my monolith's anymore … because:

First, I fetch the RDS ip address (considering it’s in a private subnet …)

data "dns_a_record_set" "db-stg" {
  host = "${aws_db_instance.db_stg.address}"
}

Then create/update our private route53 zone address with the new ip_address

data "aws_route53_zone" "dev-primary" {
    name = "dev.tikal.io."
}
resource "aws_route53_record" "db-stg" {
  zone_id = "${data.aws_route53_zone.dev-primary.zone_id}"
  name    = "db-stg.dev.tikal.io"
  type    = "A"
  ttl     = "300"
  records = ["${data.dns_a_record_set.db-stg.addrs}"]
}

And sit back for ~10/15 minuets and your good to go !

The full code looks like this:

# variables.tf ...
# Provider
variable "profile" {
  description = "your default / custom aws profile see your ~/.aws/credentials file for details"
  default     = "tikal"
}
variable "region" {
  description = "your default aws region"
  default     = "eu-west-1"
}
# RDS
variable "prod_rds_common_name" {
  default = "tikal"
}
# main.tf
provider "aws" {
  region  = "${var.region}"
  profile = "${var.profile}"
}
data "aws_vpc" "prod" {
  tags {
    Name = "tikal-default"
  }
}
data "aws_subnet_ids" "in_main_vpc" {
  vpc_id = "${data.aws_vpc.prod.id}"
}
# Get latest snapshot from production DB
data "aws_db_snapshot" "latest_db_snapshot" {
    most_recent = true
    db_instance_identifier = "${var.prod_rds_common_name}"
}
data "aws_security_group" "db-access" {
  filter {
    name   = "group-name"
    values = ["db-access"]
  }
  filter {
    name   = "vpc-id"
    values = ["${data.aws_vpc.prod.id}"]
  }
}
# Create new staging DB
resource "aws_db_instance" "db_stg" {
  instance_class       = "db.t2.large"
  identifier           = "db-stg"
  availability_zone = "${var.region}b"
  snapshot_identifier  = "${data.aws_db_snapshot.latest_db_snapshot.id}"
  vpc_security_group_ids = ["${data.aws_security_group.db-access.id}"]
  skip_final_snapshot = true
}
data "dns_a_record_set" "db-stg" {
  host = "${aws_db_instance.db_stg.address}"
}
data "aws_route53_zone" "dev-primary" {
    name = "dev.tikal.io."
}
resource "aws_route53_record" "db-stg" {
  zone_id = "${data.aws_route53_zone.dev-primary.zone_id}"
  name    = "db-stg.tikal.io"
  type    = "A"
  ttl     = "300"
  records = ["${data.dns_a_record_set.db-stg.addrs}"]
}
# outputs.tf
output "rds_addrs" {
  value = "${data.dns_a_record_set.db-stg.addrs}"
}
output "aws_db_snapshot_id" {
  value = "${data.aws_db_snapshot.latest_db_snapshot.id}"
}
output "vpc_id" {
    value = "${data.aws_vpc.prod.id}"
}

Hope you enjoyed reading this.

Until next time.

HP

Thank you for your interest!

We will contact you as soon as possible.

Send us a message

Oops, something went wrong
Please try again or contact us by email at info@tikalk.com