Tuesday, July 7, 2020

Conclusion - CI/CD Series

In this series, we have covered how to implement CI/CD (continuous integration, continuous deployment) using DevOps (development and operations). We have shown how to go from a simple change in the code, to adding tests, building an assembly, planning the deployment, and eventually apply the changes to our live environment:
  1. Introduction
  2. Base Application
  3. Unit Tests
  4. Assembly
  5. PATs
  6. Plan
  7. Apply
Future enhancements to this pipeline could include:
  • Separate DEV and PROD environments
    • DEV deploy on pushes to master
    • PROD deploy on specific tag/release cycles
  • Performance tests for the REST service
  • Ability to destroy the resources using the Terraform destroy command
  • Run PATs against service deployed in the DEV environment
    • Currently, PATs are only run against the service running in-memory

Monday, July 6, 2020

Apply - CI/CD Series

In the final piece of our CI/CD Series puzzle, we will perform the Terraform apply action. This action will perform the actual changes (as shown on the Terraform plan action) to our Heroku app and ensure everything starts up correctly. Once this command finishes, our application will be live in our account and accessible for utilization.

The Terraform apply CI stage is very similar to the plan stage we developed previously. However, there are some differences to note:
  • We will run this command on pushes to the master branch
    • The plan action was ran on pull requests to master
  • We will create a GitHub Release with a specific version of our application
    • The plan action used a hard-coded URL
  • We will point Heroku to the GitHub Release to ensure the correct version is deployed
Create Release

To create a release, we can utilize the GitHub Actions create release template. Our release name will be our application version plus the build number to ensure each release has a unique id. Once we create the release, we will save the version to a file so it can be referenced from other jobs within our CI/CD pipeline.

create_release:
runs-on: ubuntu-latest
steps:
- name: Checkout Repo
uses: actions/checkout@v2

- name: Create Release
id: release
uses: actions/create-release@v1
env:
# This token is provided by Actions, you do not need to create your own token
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: 0.1.0-${{ github.run_number }}
release_name: Release 0.1.0-${{ github.run_number }}
draft: false
prerelease: false

# Heroku needs the .tar.gz URL so modify tag URL to expected format
- name: Create Version File
run: |
export RELEASE_URL=${{ steps.release.outputs.html_url }}
RELEASE_URL+=".tar.gz"
echo "Release URL:"
echo ${RELEASE_URL}
export ARCHIVE_URL=$(echo "$RELEASE_URL" | sed 's~releases/tag~archive~')
echo "Archive URL:"
echo ${ARCHIVE_URL}
echo ${ARCHIVE_URL} >> archive.txt

# Upload version file as build artifact
- name: Upload Version File
uses: actions/upload-artifact@v2
with:
name: archive.txt
path: archive.txt

Pass Release Version

On the stage to perform the apply, we first need to read the version file uploaded when creating the release. Then, we can tell Terraform about this variable so it gets injected at runtime (since it changes on every build). In our example, we expose the variable "build_url" from our Terraform file. 

variable "build_url" {
type = string
}

# Build code & release to the app
resource "heroku_build" "guestbook_build" {
app = heroku_app.guestbook_app.name
buildpacks = ["https://github.com/heroku/heroku-buildpack-scala"]

source = {
url = var.build_url
}

To change this at runtime, we make use of Terraform's variables. This variable gets initialized by:
  1. Reading the version URL from the artifact created via the release.
  2. Exporting the version URL to an environment variable.
export TF_VAR_build_url=$(cat archive.txt)
echo "Archive URL:"
echo ${TF_VAR_build_url}

Apply Action

Now that we have everything setup, we just need to perform the actual apply command via Terraform. In this example, we still running the commands validate and plan just to ensure things are correct, but these can be skipped as we ran them on the pull request itself.

deploy:
runs-on: ubuntu-latest
needs: create_release

steps:
- name: Checkout Repo
uses: actions/checkout@v2

# Download artifact
- name: Download Version File
uses: actions/download-artifact@v2
with:
name: archive.txt

- name: Setup Terraform
uses: hashicorp/setup-terraform@v1
with:
cli_config_credentials_token: ${{ secrets.TF_API_TOKEN }}

- name: Terraform Init
id: init
run: terraform init

- name: Terraform Validate
id: validate
run: terraform validate -no-color

# The build_url is blank for planning since we will create a new URL upon commits
- name: Terraform Plan
id: plan
run: |
export TF_VAR_build_url=$(cat archive.txt)
echo "Archive URL:"
echo ${TF_VAR_build_url}
export HEROKU_API_KEY=${{ secrets.HEROKU_API_KEY }}
export HEROKU_EMAIL=${{ secrets.HEROKU_EMAIL }}
terraform plan -no-color

- name: Terraform Apply
id: apply
run: |
export TF_VAR_build_url=$(cat archive.txt)
echo "Archive URL:"
echo ${TF_VAR_build_url}
export HEROKU_API_KEY=${{ secrets.HEROKU_API_KEY }}
export HEROKU_EMAIL=${{ secrets.HEROKU_EMAIL }}
terraform apply -auto-approve -no-color

Deployed

Once the CI/CD pipeline succeeds on the master branch, the service will be live and available to be used. Also, since we used the Terraform Cloud for our remote state storage, you can browse to see how the state file has changed. This will keep track of all the changes to your application over the history of every deploy.

The live service can be accessed via its health check:
This URL is also output from the apply action:

Conclusion

In conclusion, we were able to fully automate our deploys on pushes to the master branch of our repo. This will ensure that each time a code change happens, the latest version gets automatically pushed to our live service.

The full changeset can be found on this pull request.