January 2021

Are you looking for a better way to manage your software development workflow? Tired of doing so many things manually or want to make sure bugs don’t get in the final product? That’s where DevOps comes into play! In this post, we will learn how to use Circle CI to improve our software development workflow!

Check out our related YouTube video below:

Creating an Account

The first thing we need to do is create an account. Head over to https://circleci.com/signup/ and create an account by using your GitHub or BitBucket account credentials. You will need to allow Circle CI to access some data in your SCM provider account of choice. This will allow Circle CI to be able to install webhooks and access data like your repository names, etc. This allows for automating processes such as build triggers when code is pushed to the repository.

Linking Your Repositories to Circle CI

You will need to link your repositories to Circle CI so builds can be triggered automatically when triggers you defined are triggered. Head over to the Circle CI Projects Dashboard and choose the user or organization you want to link to Circle CI. This will redirect you to a list of repositories that Circle CI is able to access under that account. For each repository, you will see a button/link that says “Set Up Project” as seen in the image below. Click on that button to start setting up your project

This will bring up a webpage that has a code editor and some menu options. You may or may not need to change these based on the specific project you chose to set up. Make sure you have chosen the proper language in the upper left above the code editor. This will sometimes change the config shown in the code editor quite a bit, so make sure you have that correctly selected before moving forward. See Below

Wrapping Up Initial Configuration

If the correct language has been selected and the selected config is showing in the editor, you can add the current config to your project to see the pipeline run. This default config will not do much except try to run unit tests. The default config will fail when it is run if you have a non-default way to run tests. You can at least see your pipeline triggered after a commit is pushed to make sure it’s hooked up correctly.

Setting Up A Basic Pipeline Config

Now let’s set up a basic pipeline config that we can actually have a pass when our Circle CI pipeline is triggered. The below example will print out a few stages in the pipeline and the last stage should be titled Print ‘Hello World’ in which it prints the words “Hello World”.

version: 2.1
orbs:
  node: circleci/[email protected]:
  build:
    docker:
      - image: circleci/node:12.20.1
    working_directory: /tmp/repo
    steps:
      - checkout
      - restore_cache:
          keys:
            - v1-dependencies-{{ checksum "package.json" }}
            - v1-dependencies-
      - run:
          name: "Install NPM Packages"
          command: "npm install"
      - run:
          name: "Print 'Hello World'"
          command: "echo 'Hello World'"
      - save_cache:
          paths:
            - node_modules
          key: v1-dependencies-{{ checksum "package.json" }}

There are many more advanced things you can add to your Circle CI pipeline. This is just the most simple example. In the next section, we will add a bit of customization to our config.

Customizing the Config File

We are going to customize our pipeline to sync all files in our project to an AWS S3 Bucket upon each commit and push. In order to do that, we will need to use a second Circle CI orb: aws-cli. To do this, add the following line to your config file:

version: 2.1
orbs:
  node: circleci/[email protected]
  aws-cli: circleci/[email protected] <<<<------ ADD THIS LINE

For the following steps to work, you will need a set of AWS IAM credentials you can use to upload files to S3. If you do not any AWS credentials, do that now by following the steps here.

Setting Up The Context

Next, we need to add what is called a context to our account. This is a set of credentials you can reference so you don’t hardcode them into your config file; which is ALWAYS a bad idea. You never want to check in files to your SCM tool with secret/sensitive values in them. So let’s add that context real quick.

Go to Circle CI Dashboard and then click Organization Settings in the left menu. Once you are in the Organization Settings menu, make sure that Contexts is selected on the left-hand side. Here is where we will create the context that holds our AWS credentials. In the main Contexts window, click the Create Contextbutton. Now give your context a name that is easy to remember. Now click on the newly created context. This will bring you to a screen that has 2 sections: Security and Environment Variables. 

Setting Up The AWS Environment Variables

In the Environment Variables section, click the Add Environment Variable button. We will add 2 environment variables to our context. First, add the variable named AWS_ACCESS_KEY_ID and then give it the value AWS_ACCESS_KEY_ID from your AWS IAM credential pair. Next, add a variable named AWS_SECRET_ACCESS_KEY and provide the corresponding value from your AWS IAM credential pair. Lastly, create an environment variable called AWS_DEFAULT_REGION and give it the value for one of the AWS regions you want to use. All of the AWS region names can be found here. Awesome! You’re done setting up your context!

Next, add a step in the build job to sync all of our files to an AWS S3 bucket. Add the following to your config file:

version: 2.1
orbs:
  node: circleci/[email protected]
  aws-cli: circleci/[email protected]
jobs:
  build:
    executor: aws-cli/default
    docker:
      - image: circleci/node:12.20.1
    working_directory: /tmp/repo
    steps:
      - checkout
      - aws-cli/setup
      - restore_cache:
          keys:
            - v1-dependencies-{{ checksum "package.json" }}
            - v1-dependencies-
      - run:
          name: "Install NPM Packages"
          command: "npm install"
      - run:
          name: "Print 'Hello World'"
          command: "echo 'Hello World'"
      - run:
          name: "Sync Files to AWS S3"
          command: aws s3 sync . s3://[bucket name goes here] --exclude "*node_modules/*"
      - save_cache:
          paths:
            - node_modules
          key: v1-dependencies-{{ checksum "package.json" }}

Now, the only thing left to do is to add a reference to the context so that the pipeline can access the AWS credentials it needs. Let’s do that now. Add the following to your config file:

version: 2.1
orbs:
  node: circleci/[email protected]
  aws-cli: circleci/[email protected]
jobs:
  build:
    docker:
      - image: circleci/node:12.20.1
    working_directory: /tmp/repo
    steps:
      - checkout
      - restore_cache:
          keys:
            - v1-dependencies-{{ checksum "package.json" }}
            - v1-dependencies-
      - run:
          name: "Install NPM Packages"
          command: "npm install"
      - run:
          name: "Print 'Hello World'"
          command: "echo 'Hello World'"
      - run:
          name: "Sync Files to AWS S3"
          command: aws s3 sync . s3://[bucket name goes here] --exclude "*node_modules/*"- save_cache:
          paths:
            - node_modules
          key: v1-dependencies-{{ checksum "package.json" }}
workflows:
  version: 2
  custom-workflow:
    jobs:
      - build:
          context: [your context name here]

Conclusion

That’s it! Your Circle CI pipeline should now auto-sync files to an AWS S3 bucket on each commit and push them to the remote repository! Now that you know how to use Circle CI, go improve your DevOps workflow!