Home » cicd
Tag:

cicd

GeneralTutorials

5 Best DevSecOps Tools in 2023

by corey.walker
written by corey.walker 9 minutes read

In today’s world, cybersecurity is constantly growing in importance for individuals as well as businesses. It is also constantly evolving and changing with different tactics and methods of exploitation by cybercriminals. However, it’s almost impossible to make sure every possible avenue of attack is fortified against exploitation. The bigger a software application becomes or the more apps & services are in use for a software development process, the bigger the opportunity for cyber criminals to find a weakness. That’s why automating as much cybersecurity into your software development process is ever-growing in importance. While automating a lot of the software development lifecycle is known as DevOps, including security-related functionality into that workflow is what’s known as DevSecOps.

DevSecOps is a great way to get a leg up on cybersecurity issues and catch them before they are available to be exploited by cybercriminals. Utilizing services and tools specific to cybersecurity within your software development lifecycle is not very difficult and can save a lot of manual effort and possible headaches from security incidents later on down the line. It just takes some upfront time to set up and configured correctly, and then just tweak things as you discover areas that need more or less stringent security based on your situation.

In this post, we will go over 5 of the best DevSecOps tools available to integrate into your software development lifecycle right now to greatly enhance your security posture for your organization without a ton of effort. They are in no particular order.

5 Best DevSecOps Tools in 2023

  1. OWASP ZAP
  2. SonarQube/SonarCloud
  3. Snyk/GitHub Dependabot
  4. Infrastructure as Code
  5. Monitoring

OWASP ZAP

OWASP is the Open Worldwide Application Security Project. It was founded on Dec 1, 2001, as a way to help individuals and businesses better secure themselves online.

OWASP annually comes out with a list of the most exploited security flaws in web applications across the entire internet. This is known as the OWASP top 10 and is used by the majority of cybersecurity personnel as a basis for what to focus on the most at any particular time. Cybersecurity professionals focus on many more areas than just these 10, but this list helps them determine what is being the most exploited security flaws at any given time.

OWASP created a piece of software called OWASP ZAP (Zed Attack Proxy) that can automatically crawl and scan a website and try multiple types of attacks against the site and report any findings it discovered may be an issue. It will try many of the same tactics used by cybercriminals when they try to attack websites. A lot of these automated tasks won’t help with more advanced attacks or attacks stemming from social engineering attacks, but they will help with more technical-related attacks.

When running this software against a website, it is possible that it can bring down your website or corrupt your data, so running it against your production website is ill-advised. How it should be used is deploying the website to a “staging” environment that is set up exactly like your production website. Make sure to NOT have this staging environment connected to production data sources but have testing data sources instead. Then run OWASP ZAP against the staging website and see what results come from the scan. If everything looks good, you can deploy changes to the production environment. If any findings are discovered by the scanner, review them and make any necessary changes, redeploy to the staging environment, scan once more, and then deploy to production once all issues are resolved.

This software is free to download and use and is open source and is a great addition to any website-related software projects.

SonarQube/SonarCloud

Whereas OWASP ZAP scans your website once it has been deployed (known as dynamic code scanning), SonarQube/SonarCloud is a product/service that will scan the source code itself before it is deployed and alert on any possible security issues related to the source code. This is known as static code scanning. It looks for things that can be exploited. Things such as not sanitizing user input which could cause things like XSS, CSRF, SQL injection, etc, or things like bad regular expressions (regex) that could cause things to consume way too many resources and cause a Denial of Service (DOS) attack, buffer overflows, etc.

This is a great addition to have in your DevOps pipeline since source code bugs have notoriously been a great source of exploited security issues. Put this hook in your SCM repository so any findings that are of a certain severity or higher will reject the change from being merged into the branch that gets pushed to production. You are guaranteed run see numerous false positives that you will have to mark as such and then run the scan again to be able to merge the code changes. But once you mark specific areas of code as “good” within SonarQube/SonarCloud, it will not flag them in subsequent scans.

This software is not free but SonarCloud can be as little as €10/month. SonarQube is software that you can license and run on your own hardware, whereas SonarCloud is Software as a Service (SaaS).

Snyk/GitHub Dependabot

Today software projects almost always have software package dependencies that are utilized to reduce the amount of code that has to be written for specific functionality. If someone else has already written something that does exactly what you need, and it doesn’t cost anything, why should someone spend the time and effort to write code that does the exact same thing? This is where things called package managers such as npm, yarn, pip, Apache Maven, etc come into play. They allow you to easily install packages that another application can easily utilize and save developers a lot of time and effort.

However, many times, these dependencies have security flaws in them that can also cause the application using them to have security issues. This is where Snyk and GitHub Dependabot come to the rescue. These services will scan all the packages that have been included in the host application and alert when there are updates available and when there are known security issues with the dependent software packages. This allows for the host application maintainers to either change the package being utilized or update the package to remediate the vulnerability. Without these automated scans and notifications, developers would have to constantly manually check all the dependent packages to see if there were any known vulnerabilities in any of them. When there are numerous projects being managed by a team and each project has numerous dependencies, that would be very inefficient and a waste of people hours.

These services have a limited free tier to use but offer paid plans for more features and more scans/repositories that can be scanned. So small teams with only a few repositories should be able to utilize the free plan(s), whereas larger organizations with more repositories or need additional scans will need to pay for the services.

Infrastructure as Code

A great way to make sure you have a consistent environment where applications are deployed is by utilizing what’s known as Infrastructure as Code. This is where all infrastructure is defined within files and stored within an SCM repository instead of being manually provisioned each time a change is needed or new infrastructure is needed. The files have everything needed declared within the files themselves and can be used multiple times to set up multiple instances the exact same way every time.

This makes it much easier to make sure everything is configured in a secure manner and replicated across every instance. It also allows for much quicker deployment of new resources when needed as each piece is not having to be done manually. Having infrastructure defined in source code files also allows for the history of changes to be tracked so that if anything negative happens after a change, it’s much easier to determine what may have caused the issue. It also gives the ability to quickly revert changes and redeploy using the previous version of the source files to a known working configuration.

There are multiple providers for Infrastructure as Code such as AWS CloudFormation, RedHat Ansible, HashiCorp Terraform, Puppet, Chef, and others. It is advised to research each to determine what is best for any given situation since each has pros and cons. Some of these also are not completely free while others are. There are also some that are specific to a particular cloud provider while others are provider agnostic.

In-Depth Monitoring

Security concerns do not stop after it’s deployed to production. Monitoring running applications is a must in order to see many issues that may not have been discovered before deploying to production.

If monitors discover a lot of outgoing traffic from your network from a particular set of IPs that have not been seen often before, it could be an indication that threat actors may be exporting a lot of private data and should be looked into immediately. Monitors may also be able to catch bugs in software that weren’t discovered in testing but are being seen by end users. This could be because initial testing didn’t think of the ways end users may either be trying to abuse the application or because how end users are just doing things not initially thought of by the development team. These monitors could show where crashes or issues are occurring that may pose a threat to security if cyber criminals were to discover and try to exploit them. This gives the chance to possibly correct these issues before any security incidents occur if the correct monitors and alerting are in place.

There are many platforms that can be utilized for monitoring and alerting. Some examples are New Relic, Datadog, AWS CloudWatch, Sentry, Dynatrace, and others. Again, these providers each have pros and cons related to pricing, offering, ad vendor lock-in. So research the options to see what may possibly be best for a given situation.

Conclusion

There are many aspects to helping to prevent cybersecurity incidents in any organization. DevSecOps is not the complete solution just as no single aspect can be. Cybersecurity has so many different aspects and avenues of attack that it takes everyone in an organization to always be thinking about it and trying to help prevent any attacks. That’s where internal company training is always needed.

However, automating as much security as possible is always a great investment to make in any organization. Having things set up to help protect you from specific types of attacks and not having to think about those particular pieces of the puzzle will greatly enhance the security posture of any organization and any software that the organization creates and customers utilize.

These may be the top 5 most important DevSecOps tools you can easily incorporate, but that doesn’t mean they should be the only ones. So now go off and secure those software development workflows!

0 comments
0 FacebookTwitterLinkedinTumblrRedditWhatsappTelegramCopy LinkPocketEmail
GitHub Actions vs Travis CI logos image
GeneralTechnology Comparisons

Which is Better? GitHub Actions vs Travis CI

by corey.walker
written by corey.walker 7 minutes read

In my previous post, I discussed the differences between GitHub Actions vs CircleCI. In earlier posts, I have also compared Travis CI vs CircleCI, Travis CI vs Jenkins, and CircleCI vs Jenkins. This post includes what is arguably the most popular newcomer to the group of CI/CD service providers, GitHub Actions. Do you know which is better for your needs? Let’s compare the two platforms and lay out the differences between each, and provide a list of what I believe are the pros and cons of each. Let’s get started!

GitHub Actions

This is an awesome new player in the CI/CD pipeline space provided by GitHub. They have been around for a long time and are also the most popular SCM provider in the world. GitHub offers great features and a great user experience for their SCM offerings. 

GitHub Actions recently joined the CI/CD pipeline space back in November 2019. It was an instant hit among many companies and people that were already using the platform. This allowed for easier integration with code already hosted on GitHub, as well as the option to reduce the number of services you need to subscribe to or purchase to have a complete CI/CD workflow. This is obviously a great feature. It also allows for much easier viewing of the pipeline status for each build since you do not have to leave GitHub’s website to see any logs and/or errors; the tab is right there as part of your repository’s UI. 

Screenshot of GitHub Actions tab in GitHub web UI

Features

You can utilize containers or VMs in your builds and also run on either GitHub.com or on-prem.

Use your own VMs, in the cloud or on-prem, with self-hosted runners.

There is support for the most popular programming languages today. These include Node.js, Python, Java, Ruby, PHP, Go, Rust, .NET, and more.

Just like Travis CI, GitHub Actions is also available to build on Linux, macOS, and Windows.

One great feature is the catalog of community-provided Actions you can add to your workflow without re-inventing the wheel. Don’t waste your precious time doing something that someone else has already done! Wondering if someone has already created the action you need in your workflow? Check out the catalog of available Actions here.

Build Matrix is another great available feature. This allows you to run multiple builds or tests simultaneously using different language versions, platform versions, etc. This can save quite a bit of time and also allow you to see if any certain language versions or platforms break with your new changes. For more information on creating a build matrix and what it can help with, check out this article.

Configuration

The configuration of GitHub Actions is very similar to that of CircleCI. You create a .github/workflows directory and create YAML files inside that directory. You can create different YAML files for each job or add them all to a single file if you wish. For a full list of available references that you can add to your config files, check out the reference guide. For a simple example of how to set up a configuration file, check out our example here.

Pricing

The pricing structure is based on paying for the amount of time your CI/CD pipelines are running per billing period. You get ~2K action minutes per month for free. Whereas Travis CI offers unlimited builds for their payment plans, they do not, however, offer a free tier except for open source projects (i.e. public repositories).

Travis CI

Travis CI is the oldest and most mature cloud-hosted CI/CD provider. It was founded in 2011 and is based in Berlin, Germany. Travis CI is among the most popular CI/CD platforms for open source projects. It has support for a variety of languages with multiple pricing plans available to suit most peoples’ needs. Travis CI is currently only available for GitHub and BitBucket.  

Configuration

Travis CI can be configured as easily as just creating a file in the root of your SCM repo called .travis.yml. You can find how to configure this file and all the available options here. This allows for the ability to copy the configuration file to multiple repositories and just edit the relevant details specific to that repo and everything will work. 


Some CI/CD tools require setting up the process manually to start with, and each time you create another project, you have to go through those same steps to set it up each time. Some steps could be overlooked or missed from time to time, which is not ideal. This can cause some of your automated processes to not work correctly, and thus possibly cause issues down the road if it goes unnoticed.

You can find steps to set up Travis CI with GitHub here. Steps to set up Travis CI with BitBucket are found here.  

Pricing


Travis CI is always free for open-source projects. For projects with private repositories, there is a free tier that offers 10K credits that once used up, you are unable to run any more builds for private repositories. This 10K credit allotment will allow for an estimated 1000 minutes of build time.

For unlimited builds for private repos that run on Windows, Linux, or FreeBSD, their pricing starts at $69/month for 1 concurrent build. For each paid plan, macOS builds now require an extra add-on of renewable credits. This means you no longer have unlimited macOS builds included in Travis CI’s paid plans. However, if you do not have any builds requiring macOS builds, or you have a limited amount of macOS builds, this may not be a huge change in pricing as $15 will get you @5K credits which will allow for roughly 500 macOS build minutes. See all details for pricing plans here.

Supported Languages

Travis CI supports a wide variety of programming languages. Currently, they support 32 different languages that are listed here and here. You can find docs for how to submit support for a new language here.  

Comparision

GitHub Actions Pros

  • One interface for both your source code repositories and your CI/CD pipelines
  • Only 1 bill to pay that includes both your SCM plan and your CI/CD service (if you pay for either now that is)
  • Build Matrix
  • Catalog of available Actions you can utilize without reinventing the wheel
  • Includes macOS builds as part of the free tier (macOS builds require 10 credits per build minute however compared to 1 credit per minute for Linux)
  • Free tier credits are renewed/reset each month 

GitHub Actions Cons

  • Newer platform. Thus not as quite many features as Travis CI as of yet (although they sure have had a strong start)
  • Only available for project repositories hosted on GitHub or GitHub Enterprise

Travis CI Pros

  • Build Matrix
  • Has a free tier (although not as many free build minutes). Private repo credits do not renew. However open-source projects are always free
  • More widely used and thus more community-based support
  • Can be hooked up to more SCMs than GitHub or GitHub Enterprise

Travis CI Cons

  • Free tier credits do not renew every month
  • Not as many available plugins to help ease the configuration of actions others have already completed
  • Need multiple subscriptions for both SCM and CI/CD services
  • Switching back and forth between CircleCI and SCM to see build logs and errors

Conclusion

The best platform for you all depends on your specific needs. If you host your code somewhere other than GitHub.com or GitHub Enterprise, you will be better off using Travis CI. However, if you host your code on GitHub.com or GitHub Enterprise, the choice can be a little more difficult. 

For most people, if they have a small number of projects whose CI/CD config files are relatively simple, I would suggest giving GitHub Actions a try. Don’t disconnect or delete your current pipelines until you try GA out for a project or two, however. Just add the config files in for GA and see how you like the flow and feel of it. Once you verify that GA has everything you need to do your builds properly and successfully, and they have been running for at least a week or two successfully, you can delete your other config files and cancel your previous CI/CD service.

If you are interested in any other CI/CD platforms and are curious about the differences in all of them, go check out my posts comparing GitHub Actions vs CircleCI, Travis CI vs CircleCI, Travis CI vs Jenkins, and CircleCI vs Jenkins.

Please let me know which platform you think is better or that you prefer using and why in the comments!

0 comments
0 FacebookTwitterLinkedinTumblrRedditWhatsappTelegramCopy LinkPocketEmail
GitHub Actions vs CircleCI logos image
GeneralTechnology Comparisons

Which is Better? GitHub Actions vs Circle CI

by corey.walker
written by corey.walker 9 minutes read

In earlier posts, I have compared Travis CI vs Circle CI, Travis CI vs Jenkins, and Circle CI vs Jenkins. In this article, I’ll be including what is arguably the most popular newcomer to the group of CI/CD service providers. Do you know which is better for your needs?

One of the newest platforms to offer the CI/CD pipeline space is GitHub Actions. Another option that has been around for a few years now and is a great candidate for any CI/CD pipelines is Circle CI. Circle CI is one of the most popular cloud-based CI/CD pipeline providers available today. In this post, I will compare the two platforms and lay out the differences between each, and provide a list of what I believe are the pros and cons of each. Let’s get started!

Check out our related YouTube video:

YouTube Thumbnail – GitHub Actions vs Circle CI

GitHub Actions

This is an awesome new player in the CI/CD pipeline space provided by GitHub. They have been around for a long time and are also the most popular SCM provider in the world. GitHub offers great features and a great user experience for their SCM offerings. 

GitHub Actions recently joined in the CI/CD pipeline space back in November 2019. It was an instant hit among many companies and people that already used the platform. This allowed for easier integration with code already hosted on GitHub, as well as the option to reduce the number of services you need to subscribe to or purchase to have a complete CI/CD workflow. This is obviously a great feature regarding GitHub Actions. It also allows for much easier viewing of the pipeline status for each build since you do not have to leave GitHub’s website to see any logs and/or errors; the tab is right there as part of your repository’s UI. 

Screenshot of GitHub Actions tab in GitHub web UI

Features

You can utilize containers or VMs in your builds and you can also use GitHub Actions to run on GitHub or on-prem.

Use your own VMs, in the cloud or on-prem, with self-hosted runners.

GitHub Actions also has support for the most popular programming languages today. These include Node.js, Python, Java, Ruby, PHP, Go, Rust, .NET, and more. Just like Circle CI, GitHub Actions is also available to build on Linux, macOS, and Windows. However, Circle CI does have a couple of extra options for building images/OS compared to GitHub Actions. 

Another great feature of GitHub Actions is the catalog of community-provided Actions you can add to your workflow without re-inventing the wheel. Don’t waste your precious time doing something that someone else has already done! Wondering if someone has already created the action you need in your workflow? Check out the catalog of available Actions here.

The build matrix is another great feature that GitHub Actions has but Circle CI lacks. This allows you to run multiple builds or tests simultaneously using different language versions, platform versions, etc. This can save quite a bit of time and also allow you to see if any certain language versions or platforms break with your new changes. For more information on creating a build matrix and what it can help with, check out this article.

Configuration

The configuration of GitHub Actions is very similar to that of Circle CI. You create a .github/workflows directory and create YAML files inside that directory. You can create different YAML files for each job or add them all to a single file if you wish. For a full list of available references, you can add to your GitHub Actions config files, check out the reference guide. For a simple example of how to set up a GitHub Actions configuration file, check out our example repo here.

Pricing

The pricing structure is similar to that of Circle CI. You pay for the amount of time your CI/CD pipelines are running per billing period. They also have a free tier that can be used indefinitely without any cost as long as you stay below the threshold for the free tier. GitHub Actions gives ~2K action minutes per month for free. 

Circle CI

Circle CI is one of the most popular cloud-hosted CI/CD providers. It was founded in 2011 and is based in San Francisco, CA. Circle CI is among the most popular CI/CD platforms for open source projects. It has support for a variety of languages with multiple pricing plans available to suit most peoples’ needs. Circle CI currently, as of this writing, is only available for repositories hosted on GitHub and BitBucket.

Configuration

You can configure Circle CI easily by just creating a file in the root of your SCM repo called .circleci/config.yml. You can find details on how to configure this file and all the available options here. This allows for the ability to copy the configuration file to multiple repositories and just edit the relevant details specific to that repo and everything will work. S

Some CI/CD tools require setting up the process manually to start with, and each time you create another project, you have to go through those same steps to set it up each time. Some steps could be overlooked or missed from time to time, which is not ideal. This can cause some of your automated processes to not work correctly, and thus possibly cause issues down the road if it goes unnoticed. You can find steps to set up Circle CI with GitHub here. You can find steps to set up Circle CI with BitBucket here.

Pricing

Circle CI is always free for open-source projects. For projects with private repositories, Circle CI has a free plan that has limited “credits” per week (as of this writing, it is 2500/week). For the default medium size Docker build container, that would give you about 250 minutes a week in build time since the medium docker image build container uses 10 credits per minute. You can use this free plan forever if your projects are not constantly running builds. If your projects only need a few builds a week, this is a great option! 

Below is a screenshot listing the majority of build container options available as of this writing and the number of credits per minute each one uses. One advantage Circle CI has over GitHub Actions is the GPU is they had GPU enhanced build option types. If your builds require the heavy lifting of GPU builds, these are a great option to have. 

Chart of credits used for different machine types
Chart of credits used for different machine types
Chart of credits used for different machine types
Chart of credits used for different machine types – Part 2

Beyond the free starter plan, Circle CI pricing is based on what they call “credits”. They do not have a paid plan that offers unlimited builds. This is something to keep in mind if you have many projects that are running builds on a consistent basis as this could spike the costs for your team if you do not estimate correctly. The paid plan starts at $30/month. You can find all pricing details for Circle CI here.

Supported Languages

Circle CI supports many of today’s most popular programming languages which can be found here.

Pros and Cons

GitHub Actions Pros

  • One interface for both your source code repositories and your CI/CD pipelines
  • Only 1 bill to pay that includes both your SCM plan and your CI/CD service (if you pay for either now that is)
  • Option to create a Build Matrix
  • You get around 2x the amount of build time per month on the free tier compared to Circle CI (2500 for GA and 1000 for Circle CI based on default configurations)
  • Catalog of available Actions you can utilize without reinventing the wheel

GitHub Actions Cons

  • Newer platform. Thus not as quite many features as Circle CI as of yet (although they sure have had a strong start)
  • Only available for project repositories hosted on GitHub or GitHub Enterprise
  • Does not have GPU-enhanced build options

Circle CI Pros

  • They have GPU-enhanced build options for projects needing the extra boot from GPU builds
  • Has a free tier (although not as many free build minutes as GitHub Actions)
  • More widely used than GitHub Actions (as of this writing) and thus more community-based support
  • Can be hooked up to more SCMs than just GitHub or GitHub Enterprise

Circle CI Cons

  • Not as many minutes on the free tier as GitHub Actions
  • Not as many available plugins to help ease the configuration of actions others have already completed
  • Need multiple subscriptions for both SCM and CI/CD services
  • Switching back and forth between Circle CI and SCM to see build logs and errors

Conclusion

The best platform for you all depends on your specific needs. If you host your code somewhere other than GitHub.com or GitHub Enterprise, you will be better off using Circle CI. However, if you host your code on GitHub.com or GitHub Enterprise, the choice can be a little more difficult. 

For most people, if they have a small number of projects whose CI/CD config files are relatively simple, I would suggest moving over to try GitHub Actions. Don’t disconnect or delete your current pipelines until you try GA out for a project or two, however. Just add the config files in for GA and see how you like the flow and feel of it. Once you verify that GA has everything you need to do your builds properly and successfully, and they have been running for at least a week or two successfully, you can delete your other config files and cancel your previous CI/CD service.

If you have complex projects or projects that need the hefty boost of GPU builds, I would suggest trying out or sticking with Circle CI. They have a few more options to help with more complex build scenarios since they are the mature CI/CD service provider. 

Go check out my posts comparing Travis CI vs Circle CI, Travis CI vs Jenkins, and Circle CI vs Jenkins to learn about other CI/CD platforms and the differences between them.

Please let me know which platform you think is better or that you prefer using and why in the comments!

0 comments
0 FacebookTwitterLinkedinTumblrRedditWhatsappTelegramCopy LinkPocketEmail
CircleCI Getting Started Image
Tutorials

How To Use Circle CI – Getting Started With CI/CD

by corey.walker
written by corey.walker 6 minutes read

Are you looking for a better way to manage your software development workflow? Tired of doing so many things manually or want to make sure bugs don’t get in the final product? That’s where DevOps comes into play! In this post, we will learn how to use Circle CI to improve our software development workflow!

Check out our related YouTube video below:

Creating an Account

The first thing we need to do is create an account. Head over to https://circleci.com/signup/ and create an account by using your GitHub or BitBucket account credentials. You will need to allow Circle CI to access some data in your SCM provider account of choice. This will allow Circle CI to be able to install webhooks and access data like your repository names, etc. This allows for automating processes such as build triggers when code is pushed to the repository.

Linking Your Repositories to Circle CI

You will need to link your repositories to Circle CI so builds can be triggered automatically when triggers you defined are triggered. Head over to the Circle CI Projects Dashboard and choose the user or organization you want to link to Circle CI. This will redirect you to a list of repositories that Circle CI is able to access under that account. For each repository, you will see a button/link that says “Set Up Project” as seen in the image below. Click on that button to start setting up your project

screenshot of CircleCI dashboard project list

This will bring up a webpage that has a code editor and some menu options. You may or may not need to change these based on the specific project you chose to set up. Make sure you have chosen the proper language in the upper left above the code editor. This will sometimes change the config shown in the code editor quite a bit, so make sure you have that correctly selected before moving forward. See Below

Dashboard of how to use Circle CI screenshot

Wrapping Up Initial Configuration

If the correct language has been selected and the selected config is showing in the editor, you can add the current config to your project to see the pipeline run. This default config will not do much except try to run unit tests. The default config will fail when it is run if you have a non-default way to run tests. You can at least see your pipeline triggered after a commit is pushed to make sure it’s hooked up correctly.

Setting Up A Basic Pipeline Config

Now let’s set up a basic pipeline config that we can actually have a pass when our Circle CI pipeline is triggered. The below example will print out a few stages in the pipeline and the last stage should be titled Print ‘Hello World’ in which it prints the words “Hello World”.

version: 2.1
orbs:
  node: circleci/[email protected]:
  build:
    docker:
      - image: circleci/node:12.20.1
    working_directory: /tmp/repo
    steps:
      - checkout
      - restore_cache:
          keys:
            - v1-dependencies-{{ checksum "package.json" }}
            - v1-dependencies-
      - run:
          name: "Install NPM Packages"
          command: "npm install"
      - run:
          name: "Print 'Hello World'"
          command: "echo 'Hello World'"
      - save_cache:
          paths:
            - node_modules
          key: v1-dependencies-{{ checksum "package.json" }}

There are many more advanced things you can add to your Circle CI pipeline. This is just the most simple example. In the next section, we will add a bit of customization to our config.

Customizing the Config File

We are going to customize our pipeline to sync all files in our project to an AWS S3 Bucket upon each commit and push. In order to do that, we will need to use a second Circle CI orb: aws-cli. To do this, add the following line to your config file:

version: 2.1
orbs:
  node: circleci/[email protected]
  aws-cli: circleci/[email protected] <<<<------ ADD THIS LINE

For the following steps to work, you will need a set of AWS IAM credentials you can use to upload files to S3. If you do not any AWS credentials, do that now by following the steps here.

Setting Up The Context

Next, we need to add what is called a context to our account. This is a set of credentials you can reference so you don’t hardcode them into your config file; which is ALWAYS a bad idea. You never want to check in files to your SCM tool with secret/sensitive values in them. So let’s add that context real quick.

Go to Circle CI Dashboard and then click Organization Settings in the left menu. Once you are in the Organization Settings menu, make sure that Contexts is selected on the left-hand side. Here is where we will create the context that holds our AWS credentials. In the main Contexts window, click the Create Contextbutton. Now give your context a name that is easy to remember. Now click on the newly created context. This will bring you to a screen that has 2 sections: Security and Environment Variables. 

Setting Up The AWS Environment Variables

In the Environment Variables section, click the Add Environment Variable button. We will add 2 environment variables to our context. First, add the variable named AWS_ACCESS_KEY_ID and then give it the value AWS_ACCESS_KEY_ID from your AWS IAM credential pair. Next, add a variable named AWS_SECRET_ACCESS_KEY and provide the corresponding value from your AWS IAM credential pair. Lastly, create an environment variable called AWS_DEFAULT_REGION and give it the value for one of the AWS regions you want to use. All of the AWS region names can be found here. Awesome! You’re done setting up your context!

Next, add a step in the build job to sync all of our files to an AWS S3 bucket. Add the following to your config file:

version: 2.1
orbs:
  node: circleci/[email protected]
  aws-cli: circleci/[email protected]
jobs:
  build:
    executor: aws-cli/default
    docker:
      - image: circleci/node:12.20.1
    working_directory: /tmp/repo
    steps:
      - checkout
      - aws-cli/setup
      - restore_cache:
          keys:
            - v1-dependencies-{{ checksum "package.json" }}
            - v1-dependencies-
      - run:
          name: "Install NPM Packages"
          command: "npm install"
      - run:
          name: "Print 'Hello World'"
          command: "echo 'Hello World'"
      - run:
          name: "Sync Files to AWS S3"
          command: aws s3 sync . s3://[bucket name goes here] --exclude "*node_modules/*"
      - save_cache:
          paths:
            - node_modules
          key: v1-dependencies-{{ checksum "package.json" }}

Now, the only thing left to do is to add a reference to the context so that the pipeline can access the AWS credentials it needs. Let’s do that now. Add the following to your config file:

version: 2.1
orbs:
  node: circleci/[email protected]
  aws-cli: circleci/[email protected]
jobs:
  build:
    docker:
      - image: circleci/node:12.20.1
    working_directory: /tmp/repo
    steps:
      - checkout
      - restore_cache:
          keys:
            - v1-dependencies-{{ checksum "package.json" }}
            - v1-dependencies-
      - run:
          name: "Install NPM Packages"
          command: "npm install"
      - run:
          name: "Print 'Hello World'"
          command: "echo 'Hello World'"
      - run:
          name: "Sync Files to AWS S3"
          command: aws s3 sync . s3://[bucket name goes here] --exclude "*node_modules/*"- save_cache:
          paths:
            - node_modules
          key: v1-dependencies-{{ checksum "package.json" }}
workflows:
  version: 2
  custom-workflow:
    jobs:
      - build:
          context: [your context name here]

Conclusion

That’s it! Your Circle CI pipeline should now auto-sync files to an AWS S3 bucket on each commit and push them to the remote repository! Now that you know how to use Circle CI, go improve your DevOps workflow!

0 comments
0 FacebookTwitterLinkedinTumblrRedditWhatsappTelegramCopy LinkPocketEmail
TravisCI vs Jenkins Logos Image
GeneralTechnology Comparisons

Which Is Better? Travis CI vs Jenkins

by corey.walker
written by corey.walker 5 minutes read

Software development and automation of the process have advanced a great deal in recent years. There are now many different platforms you can utilize to automate your software development workflow. Utilizing one of these CI (Continuous Integration) and CD (Continuous Delivery) tools is a best practice for the software development lifecycle. It helps automate things when new changes are made. Do you know which is better for your needs? Do you know the differences between Travis CI vs Jenkins?

In this post, I will compare the two platforms and lay out the differences between each, and provide a list of what I believe are the pros and cons of each. Let’s get started!

Check out our related YouTube video:

YouTube video for Travis CI vs Jenkins

Travis CI

Founded in 2011, Travis CI is the oldest and most mature cloud-hosted CI/CD provider. They are based in Berlin, Germany. Travis CI is among the most popular CI/CD platforms for open source projects. It has support for a variety of languages with multiple pricing plans available to suit most peoples’ needs. Travis CI is currently only available for GitHub and BitBucket.

Configuration

Travis CI can be configured by just creating a file in the root of your SCM repo called .travis.yml. You can find details on how to configure this file and all the available options here. This allows for the ability to copy the configuration file to multiple repositories and just edit the relevant details specific to that repo and everything will work.

Some CI/CD tools require setting up the process manually to start with, and each time you create another project, you have to go through those same steps to set it up each time. That is not really ideal as some steps could be overlooked or missed from time to time. This can cause some of your automated processes to not work correctly, and thus possibly cause issues down the road if it goes unnoticed.

You can find steps to set up Travis CI with GitHub here. Steps to set up Travis CI with BitBucket are found here.

Pricing

Travis CI is always free for open-source projects. For projects with private repositories, your first 100 builds are free in order to allow you to try it out even for private repositories. For unlimited builds for projects with private repos, there are a variety of plans for developers and teams of all sizes starting at $65/month as of this writing (June 2020). You can find the full list of plan options for private repositories here.

Supported Languages

Travis CI supports a wide variety of programming languages. They support 32 different languages that are listed here and here. You can find the docs for how to submit support for a new language here.

Jenkins

Jenkins first started back in 2005 which makes it one of the oldest CI/CD platforms available. It has been installed millions of times. This makes it one of the most, if not THE most, popular CI/CD platforms in use today. It has over 1500 plugins to choose from to customize your CI/CD needs. Jenkins is open-source and does not cost anything for the software itself. The cost of infrastructure needed to run it is the only cost.

Jenkins is only available on-premise except through third-party providers. They do not have a Jenkins-hosted cloud-based CI/CD service that you can subscribe to. There are some offerings such as Google Cloud’s Jenkins Cloud offering and Cloudbees Jenkins Platform that allow you to use a cloud-based Jenkins solution, however.

Jenkins allows for simultaneous builds running concurrently by default if you have the infrastructure to support it. Numerous large companies use Jenkins for their CI/CD needs.

Although Jenkins does not have a cloud-based offering directly from Jenkins, there are a few hosting providers that offer such an offering. However, one of the major draws of Jenkins for users is the customizability of using the multitude of plugins available.

Pros and Cons

Travis CI Pros

The benefits of Travis CI over Jenkins are the following:

  • Cloud-based offering
  • No need to pay for, configure or maintain your own infrastructure
  • Build Matrices

Travis CI Cons

  • In order to get multiple concurrent builds, the monthly price can get expensive quickly
  • No completely free account for projects with private repositories
  • Not as many plugins available that can simplify the customization of CI/CD flow compared to Jenkins
  • Only available for repositories stored in GitHub or BitBucket

Jenkins Pros

  • Open-source software, well-vetted
  • Over 1,500 plugins to help simplify and customize CI/CD flow
  • Completely free software, only need to pay for the infrastructure you want to allocate
  • More widely used than Travis CI and thus more community-based support
  • More SCMs than just GitHub and BitBucket

Jenkins Cons

  • No cloud-based subscription offering directly from vendors themselves. Requires third-party platforms hosting Jenkins in the cloud
  • Have to pay for, configure, and maintain your own infrastructure to run Jenkins
  • With the multitude of plugins available, it can be a bit of a learning curve when learning which ones are best for your particular situation

Conclusion

The best platform for you all depends on your specific needs. If you do not have a large monthly budget or extra man-hours to spend on infrastructure issues, then a cloud-based provider such as Travis CI will be a better choice. Just note that Travis CI only integrates with GitHub and Bitbucket as I write this (June 2020).

If you are building projects that contain repositories stored somewhere other than GitHub or BitBucket, you would probably be better off using Jenkins for now until Travis CI expands its SCM integrations. If you are on a budget, try one of the third-party cloud-based Jenkins offerings I referenced earlier in this story.

You can also check out my post comparing Travis CI vs CircleCI and my comparison between CircleCI vs Jenkins.

Please let me know which platform you think is better or that you prefer using and why in the comments!

0 comments
0 FacebookTwitterLinkedinTumblrRedditWhatsappTelegramCopy LinkPocketEmail
TravisCI vs CircleCI Logos Image
GeneralTechnology Comparisons

Which Is Better? Travis CI vs Circle CI

by corey.walker
written by corey.walker 6 minutes read

Software development and automation of the process have advanced a great deal in recent years. There are now many different platforms you can utilize to automate your software development workflow. Utilizing one of these CI (Continuous Integration) and CD (Continuous Delivery) tools is a best practice for the software development lifecycle. It helps to automate things when new changes are added. Do you know which is better for your needs? Do you know the differences between Travis CI vs Circle CI?

In this post, I will compare the two platforms and lay out the differences between each, and provide a list of what I believe are the pros and cons of each. Let’s get started!

Check out our related YouTube video:

YouTube video for Travis CI vs Circle CI

Travis CI

Founded in 2011, Travis CI is the oldest and most mature cloud-hosted CI/CD provider. They are based in Berlin, Germany. They are among the most popular CI/CD platforms for open source projects. It has support for a variety of languages with multiple pricing plans available to suit most peoples’ needs. It is currently only available for GitHub and BitBucket.

Configuration

Configuration is as easy as just creating a file in the root of your SCM repo called .travis.yml. You can find details on how to configure this file and all the available options here. This allows for the ability to copy the configuration file to multiple repositories and just edit the relevant details specific to that repo and everything will work.

Some CI/CD tools require setting up the process manually to start with, and each time you create another project, you have to go through those same steps to set it up each time. That is not really ideal as some steps could be overlooked or missed from time to time. This can cause some of your automated processes to not work correctly, and thus possibly cause issues down the road if it goes unnoticed.

You can find steps to set up Travis CI with GitHub here and steps to set up with BitBucket here.

Pricing

Always free for open-source projects. For projects with private repositories, your first 100 builds are free in order to allow you to try it out even for private repositories. For unlimited builds for projects with private repos, there are a variety of plans for developers and teams of all sizes starting at $65/month as of this writing (June 2020). You can find the full list of plan options for private repositories here.

Supported Languages

They support a wide variety of programming languages. They support 32 different languages that can be found here and here. You can find the docs on how to submit support for a new language here.

They does not charge extra for access to macOS build environments. Circle CI does not charge “extra” per se, but requires more “credits” per minute than Linux build environments and thus can eat through your credits much faster than expected.

Circle CI

Also started in 2011, Circle CI has been around as long as Travis CI. They do not have quite as large of a customer base as Travis CI but are an equally powerful CI/CD platform that is based in San Francisco, CA. It also has support for a variety of languages with multiple pricing plans available to suit most peoples’ needs just as Travis CI does. Circle CI also is currently only available for GitHub and BitBucket.

Configuration

Configuration for Circle CI is just as easy by creating a directory in the root of your SCM repo called .circleci. Inside this directory, you create a file config.yml. You can find details on how to configure this file and all the available options here. This allows for the ability to copy the configuration file to multiple repositories and just edit the relevant details specific to that repo and everything will work just like Travis CI.

You can find steps to set up Circle CI with GitHub here. Steps to set up Circle CI with BitBucket are found here.

Pricing

Circle CI is always free for open-source projects. For projects with private repositories, you have multiple plans available with a Free plan available with limited credits each week. So if you have a project that doesn’t require a lot of changes and thus not many builds each week, this could be a permanent free solution.

Unlike Travis CI where you buy different plans which are all unlimited builds and build minutes but only differ by the number of concurrent builds, Circle CI plans are based on “credits”. This can allow you to auto-scale to the capacity of concurrent builds as needed by using more credits for each additional concurrent build. Pricing starts at $30/month which includes 3 users and 25,000 credits with each additional 25,000 credits costing $15 and an additional $15 per additional user.

Circle Ci charges more credits per minute for macOS builds than for the Linux Docker builds. The differences between normal Linux VM and macOS are minimal.

Supported Languages

Circle CI also supports a wide variety of programming languages. You can find information on Circle CI’s supported languages here.

Pros and Cons

Travis CI Pros

  • Unlimited builds for each paid plan
  • More languages supported by default
  • Build Matrices

Travis CI Cons

  • Higher starting price point for paid plans
  • No completely free account for projects with private repos
  • No concurrent builds until you pay more per month for higher-priced plans

Circle CI Pros

  • Completely free account for projects with private repos with limited build minutes each week
  • Cheaper starting price point for paid plans
  • Can automatically have multiple concurrent builds at the lowest price point

Circle CI Cons

  • Paid plans are based on “credits”. Credits are spent for each minute of build time
  • Credits are used at different rates depending on OS and VM size. Thus if you are building on Windows or macOS, you spend more credits per minute than Docker or Linux VM builds.

Conclusion

The best platform for you all depends on your specific needs. If you are only building projects that only have open source repositories, then either platform will be free. Thus, it’s all based on personal preference.

If you are building projects that contain private repositories and are heavily utilizing Windows or macOS VMs, it would most likely be cheaper, in the long run, to go with Travis CI. Circle CI will be cheaper if you have projects that are not updated very frequently and can be built on Linux, or if you can mostly utilize Circle Ci’s Docker Linux builds.

The other major thing to consider is the language being used for your projects. If one platform offers it and the other platform doesn’t, then the platform that does offer it would probably suit your needs better.

You can also check out my comparison between Travis CI and Jenkins or Circle CI and Jenkins.

Please let me know which platform you think is better and why in the comments!

0 comments
0 FacebookTwitterLinkedinTumblrRedditWhatsappTelegramCopy LinkPocketEmail
AWS Cloudwatch and Route53 logos image
Tutorials

How To Setup Multi-Region DNS Fail-over In AWS

by corey.walker
written by corey.walker 7 minutes read

Do you have an API hosted on AWS API Gateway? Do you wonder how you can deploy that API into multiple regions around the globe and have automatic fail-over protection in case one region goes offline so that your API keeps on working without any intervention? Well, you came to the right blog post. In this post, I will show you how to set up and configure your AWS Route53 DNS entries, health checks, and AWS API Gateway APIs in order to have that automatic multi-region DNS fail-over protection you’re looking for.

Initial Multi-Region DNS Fail-over Configuration

The first thing we need to do is have a domain setup in AWS Route 53. Once you have a domain, we also need to set up a hosted zone in Route 53 for that domain. Inside this hosted zone is where we will add the DNS records we need as part of our automatic DNS fail-over configuration. Once you have a domain name and hosted zone setup, let’s get going on configuring automatic DNS fail-over!

First things first, we need to have the desired API deployed to AWS API Gateway in each of the AWS regions we want in our setup. If you do not have that yet, you can follow this guide as an example of how to create an API using HTTP front-end with AWS Lambda as the back-end for processing the requests. You will need an endpoint on your API that will respond to GET requests and be able to return a status code of 2XX in order for our health checks to know that our API is healthy in each region.

Creating An SSL/TLS Certificate

Once you have that, you will need to create an SSL certificate on AWS Certificate Manager in each of the same regions to use with your domain which we will use in the next step. You will need to verify the entries you add to your SSL certificate. If you are using Route 53 as we have discussed in this article, you can expand each of the domain name listings and add it to Route 53. We then just need to wait on the verification to be complete before moving on to the next step.

Now that you have your API deployed to AWS API Gateway and we have an SSL certificate for our domain, we need to set up a custom domain name in AWS API Gateway in each of our AWS regions. Go to the AWS API Gateway console and click Create Custom Domain Name. It will open up a form to create a new custom domain which you should fill out as follows:

  • For Domain Name, fill in your desired domain for this API. (e.g. examplecorp.com)
  • Choose HTTP Protocol
  • Security Policy: TLS 1.2 (1.2 is the latest and most secure as of this writing. If there is a newer version as you’re reading this, choose it instead)
  • Endpoint Configuration: Regional (this selection is important!)
  • Choose the SSL certificate you created in the previous steps, then click Save

Cool. Now we have everything configured in API Gateway. Let’s move on to Route 53 to finish up our automatic multi-region DNS fail-over!

Configuring Health Checks

We now need to set up the health checks that allow us to automatically determine if a certain region is unavailable. We will use this to route our traffic accordingly. Visit the AWS Route 53 Console then click on Health Checks in the left-hand menu and then click Create health check.

Under the first section under Configure health check, give this health check a name that would specify the region, service, and environment this is being used for so we can keep things organized and easy to know what health check is monitoring what. A good name may be something like Production — US-East-1 — API for example. For What to monitor, select Endpoint.

Monitor Endpoint Configuration

Now, in the Monitor an endpoint section, use the following configuration:

  • Specify endpoint by: Domain Name
  • Protocol: HTTPS (this is required to be HTTPS because we use API Gateway API stage generated URLs which are HTTPS)
  • Domain Name: This will be the Invoke URL generated for the API stage in API Gateway such as dev, prod, etc after deploying an API. Template URL for where you can find it is https://console.aws.amazon.com/apigateway/home?region=us-east-1#/apis/{{API_ID}}/stages/dev and an example URL is dk0bnnfwn2.execute-api.us-east-1.amazonaws.com
  • Leave Port as default since it is determined by the protocol you selected
  • For Path, enter the stage name for the URL you copied above (e.g. dev, production, etc) followed by the endpoint path you want the health check to monitor. (The one I mentioned earlier that needed to have a GET method and return a 2XX status code). For example, a valid path could be dev/health

You can expand the Advanced Configuration section and change some of the options if you wish. One I would possibly really consider modifying is the list of Health Checker Regions. You have to choose at least 3 regions, choose 3 for development APIs, and additional for my production APIs.

CloudWatch Alarms

Click Next and on the next screen, you can set up CloudWatch alarms when the health check fails for the number of consecutive times specified in the Advanced Configuration section on the first page. Continue on and create the health check.

Repeat the above steps for each of the regions you wish to deploy your API, remembering to grab the correct Invoke URL for the API in each region. Now you should have all of your health checks in place. Let’s move on to the last piece of the puzzle, the DNS entries in your hosted zone for your domain.

Creating Route53 Record Sets

Let’s get the last piece configured now. Head over to the AWS Route 53 Console once again, and click on Hosted Zones in the left menu, then click on the hosted zone for your domain. Click on Create Record Set and enter the following details:

  • Name: desired domain/subdomain mapping you want for this API (e.g. api.examplecorp.com).
  • Record type: CNAME
  • Alias: No
  • TTL: 300 (5m)
  • Value: This is the value of Target Domain Name generated when you created the custom domain in API Gateway. Copy that value and paste it here
  • Routing Policy: Latency
  • Select the region that matches the custom domain Target Domain Nameyou just pasted, and then give this policy a name that distinguishes it from the other regions and environments you may use.
  • Associate With Health Check: Yes then select the health check that corresponds with the same region and environment that matches the other values for this CNAME record
  • Click Create
  • Follow the same steps for all other regions you deployed your API to

Conclusion

That’s it! Now whenever one of your regions is unavailable for whatever reason, it will take just a few minutes (<5 minutes) for your other regions to pick up the load. The response times may be a little slower than the desired region since these are latency-based routing rules. However, your application should keep on working after the health checks fail the specified number of times and fail-over occurs! Congratulations! You have now set up automatic multi-region DNS fail-over using AWS Route53 and API Gateway!

0 comments
0 FacebookTwitterLinkedinTumblrRedditWhatsappTelegramCopy LinkPocketEmail
Serverless Framework Tutorials Image
Tutorials

How To Deploy Multiple Environments Using Serverless

by corey.walker
written by corey.walker 4 minutes read

Do you currently use a serverless configuration for any applications or interested in migrating a current application to use serverless? Have you wondered how you can easily configure your CI/CD pipeline to easily deploy to multiple environments such as development, staging, and all production environments? With the Serverless Framework you can do just that. Continue reading to find out how.

Intro To Serverless Framework

Currently, I use the Serverless Framework to deploy to multiple environments and regions on the AWS cloud. I deploy to our development, staging, and production environments. Our development and staging environments are deployed in 2 different regions. Our production environment is deployed to 4 regions. So I have a serverless.yml file with many properties found in the docs for serverless.yml found here.

Configuration

  • All properties that remain the same for all environments and/or regions go into the serverless.yml base file
  • For all properties that are different between each environment such as development, staging, and production, I have another config file with the following naming convention under my config/ directory (the same naming I have is not required, but just what i use):
  • config/config.dev.yml, config/config.stg.yml, config/config.prod.yml, etc

Following this guide, I add all the needed custom properties to those environment-specific config files, and then in the main serverless.yml file, I create the same property but then reference the property from the environment/stage specified on the serverless deploy -s {stage}CLI command. I use a combination of both the stage CLI parameter and setting an ENV variable in a bash script before calling serverless deploy. Then I can also use ${env:ENV} to reference that value inside serverless.yml .

Environment Specific Configurations

Here’s an example of an environment-specific YAML configuration file I have for environment-specific variables:

stageSuffix: -${self:provider.stackSuffix, 'dev'}
lambda:
  versioning: false
prune:
  automatic: ${self:lambda.versioning}
  number: 5
api:
  domain: api.dev.${self:custom.domain}
storage:
  bucket: dev.${self:provider.region}.${self:custom.domain}
slackChannel: 'dev-slack-channel'

And inside serverless.yml, I reference those values by doing the following:

environment:
  ENV: ${env:ENV, 'dev'}
  STAGE: ${opt:stage, 'dev'}
  REGION: ${opt:region, 'us-east-1'}
  SLACK_CHANNEL: ${file(./config/config.${self:provider.stage}.yml):slackChannel}

Also, using a lot of shell environment-based variables allows you to make your serverless config even more decoupled away from specific projects/microservices since you then can just create a .env file specific to each project, create a bash script to run your serverless deploycommands, and before running serverless deploy, run source .env and many values can be populated easily by just supplying your .env file

Lastly, here’s an example of the bash script we use to run our serverless deploy commands that will supply many of our values in serverless.ymlwith values from a .env file instead of manually finding and replacing all the values between all our different microservices:

set -evif [[ -e "./secret.env" ]]; then
    source "./secret.env"
fi
if [[ -e "./.env" ]]; then
    source "./.env"
fiBRANCH="${CIRCLE_BRANCH:-$BRANCH}"if [[ $# -gt 0 ]]; then
    REGION="$1"
fiif [[ -n "$REGION" ]]; then
    REGION="$REGION"
else
    REGION="us-east-1"
fiif [[ "$BRANCH" == "development" ]] || [[ "$BRANCH" =~ ^issue-* ]]; then
    ENV="dev"
elif [[ "$BRANCH" == "staging" ]]; then
    ENV="stg"
elif [[ "$BRANCH" == "master" ]]; then
    ENV="prod"
fiif [[ "$BRANCH" =~ ^issue-* ]]; then
    export STAGE="feature"
    export STACK_SUFFIX="$BRANCH"
    export API_BASE_PATH_SUFFIX="-$BRANCH"
    "$(dirname "$0")"/create_domain.sh -s "$STAGE" -r "$REGION"
    "$NODE_MODULES_CLI_DIR"/serverless deploy -v -s "$STAGE" -r "$REGION"
else
    export STACK_SUFFIX="$ENV"
    "$(dirname "$0")"/create_domain.sh -s "$ENV" -r "$REGION"
    "$NODE_MODULES_CLI_DIR"/serverless deploy -v -s "$ENV" -r "$REGION"
fi

and the .env file we use to supply the variable values:

export BRANCH="$(git branch | grep \* | cut -d ' ' -f2)"
export ENV="dev"
export STACK_PREFIX="{STACK_PREFIX_HERE}"
export STACK_SUFFIX="$ENV"
export APP="{APP_ID_HERE}"
export APP_NAME="{APP_NAME_HERE}"
export COMPONENT="{COMPONENT_NAME_HERE}"
export REGION="us-east-1"
export DYNAMODB_ENDPOINT="dynamodb.$REGION.amazonaws.com"
export GITHUB_USERNAME="{GITHUB_USERNAME_HERE}"
export DB_TABLE_NAME="$COMPONENT"
export DOMAIN="{DOMAIN_NAME_HERE}"
export EMAIL_SUPPORT="support@${DOMAIN}"
export NODE_PRESERVE_SYMLINKS=1
export TEST_DEPLOYMENT_SUFFIX="-test"
export SLACK_CHANNEL="slack-channel-${ENV}"
export API_VERSION="v1"
export API_BASE_PATH=""
export API_BASE_PATH_PREFIX=""
export API_BASE_PATH_SUFFIX=""
export NODE_MODULES_CLI_DIR="./node_modules/.bin"

Conclusion

Let me know what you think and if you also see any improvement I myself could make to this process!

0 comments
0 FacebookTwitterLinkedinTumblrRedditWhatsappTelegramCopy LinkPocketEmail
Require Successful Circle CI builds for GitHub PRs
Tutorials

How To Require Circle CI Builds For Pull Requests

by corey.walker
written by corey.walker 4 minutes read


You are a software developer and/or a project manager. You and your team want to make sure that new code changes don’t break your project before pushing to production. Great! Well, how do we go about making sure that we do everything we can to make sure that doesn’t happen? I’ll walk you through how to require successful Circle CI builds before pull requests can be merged!

One great thing to add to you workflow is requiring successful builds with the new code before merging. If you are using GitHub and Circle CI, I am going to lay out how you can leverage the GitHub’s status API to block pull requests if new code breaks your builds.

Setup GitHub And Circle CI

First thing we need to setup is GitHub. If you don’t already have a repository setup in GitHub, do that first thing. Next, you’ll want to head over to Circle CI and sign in with GitHub. Once you are logged in to Circle CI, you should see a screen similar to the following:

Circle CI dashboard showing projects
Circle CI dashboard showing projects

Setting Up Configuration File

Next, click Set Up Project next to the repository you wish to setup automated builds with. This will show a menu of the different default programming languages that Circle CI integrates with and has default images for. It will look like the following:

OS and language options in Circle CI
OS and language options in Circle CI

Select your desired programming language, which will then show a default base configuration file for that particular language. For example, if you select Node as your desired programing language, it will show you a default Circle CI configuration file that resembles the following:

# Javascript Node Circle CI 2.0 configuration file
#
# Check https://circleci.com/docs/2.0/language-javascript/ for more details
#
version: 2
jobs:
  build:
    docker:
      # specify the version you desire here
      - image: circleci/node:7.10

      # Specify service dependencies here if necessary
      # CircleCI maintains a library of pre-built images
      # documented at https://circleci.com/docs/2.0/circleci-images/
      # - image: circleci/mongo:3.4.4

    working_directory: ~/repo

    steps:
      - checkout

      # Download and cache dependencies
      - restore_cache:
          keys:
            - v1-dependencies-{{ checksum "package.json" }}
            # fallback to using the latest cache if no exact match is found
            - v1-dependencies-

      - run: yarn install

      - save_cache:
          paths:
            - node_modules
          key: v1-dependencies-{{ checksum "package.json" }}      # run tests!
      - run: yarn test

In your repository on GitHub, create a file .circleci/config.yml in your master (or current default)branch and paste the example config file into that file. After you have done that, go back to Circle CI and in the setup screen where you copied the default config file from, click the button Start Building. If your first build fails, it may be because you don’t have any unit tests setup yet. You can remove the - run: yarn test line and replace it with — run: echo "Hello World" in order just to get the build passing.

After getting your first successful build on Circle CI, we need to go back to GitHub and go to your repo, then click Settings in top menu list > Branches in left menu list. Next to Branch Protection Rules, click Add Rule. This will bring up a screen that looks like the following:

GitHub repository branch protection rules
GitHub repository branch protection rules

As you can see from the above photo, you will need to select Require status checks to pass before merging option. This is what we need in order to block any PRs from being merged if builds fail with new changes. If you also want to make sure the new changes being merged aren’t missing any code that has been pushed since the current changes have been checked out, you’ll also want to select Require branches to be up to date before merging. As you can also see, you will need to select ci/circleci: build to be one of the statuses that has to be successful before merging.

Conclusion

After applying these changes to GitHub, any future pull requests for the repository and for the specified branch(es). This should help keep errors from reaching your production environments.

0 comments
0 FacebookTwitterLinkedinTumblrRedditWhatsappTelegramCopy LinkPocketEmail