TL;DR: In this article, you will learn how to configure a Continuous Deployment pipeline for your open-source web apps. For demonstration purposes, you will use Now.sh, GitHub, and Travis CI to automate the pipeline. However, the strategy can actually be used with other programming languages (e.g. Python, Java, and .NET Core) and tools (like BitBucket, AWS, and CircleCI).

"Learn how to configure a Continuous Deployment pipeline for your open-source web applications."

Continuous Deployment Overview

Continuous Deployment (popularly known as CD) is a modern software engineering approach that has to do with automating the release of software. Instead of the usual manual method of pushing out software to production, Continuous Deployment aims to ease and automate this process with the use of pipelines. In Continuous Deployment, an update to the source code means an update to the production server too if all tests are passed. Continuous Deployment is often mistaken with Continuous Integration and Continuous Delivery. For you to properly get a hang of this concept, let us distinguish the other two concepts.

In Continuous Integration (CI), when a new code is checked in, a build is generated and tested. The aim is to test every new code to be sure that it doesn’t break the software as a whole. This will require writing test for every update that is pushed. The importance of CI is to ensure a stable codebase at all times, especially when there are multiple developers in a team. With this, bugs are discovered more rapidly when the automated tests fail.

Continuous Delivery moves a step ahead of Continuous Integration. After testing, the release process is also automated. The aim is to generate a releasable build (i.e. a build that is stable enough to go into production). This helps to reduce the hassle of preparing for release. In Continuous Delivery, since there are regular releases, there is also faster feedback.

The major difference between Continuous Delivery and Continuous Deployment is the way releases are done. One is manual, while the other is automated. With Continuous Delivery, your software is always at a state where it can be pushed out to production manually. Whereas, with Continuous Deployment, any stable working version of the software is pushed to production immediately. Continuous Deployment needs Continuous Delivery, but the reverse is not applicable.

In all of these, you need a repository where your code will reside and a Continuous Integration server to monitor the repository where the code is checked into. When the server notices an update in the code, it triggers the pipeline. A pipeline in this context is a script/file that contains commands and tasks to be executed on the project. When typically configuring your Continuous Integration server, you set up your pipeline alongside it. Some common Continuous Integration servers include Travis CI, Jenkins, TeamCity, etc.

In this post, you will learn how to set up a Continuous Integration server together with a GitHub repository to show Continuous Deployments in practice. Continuous Deployments are important for the following reasons: better integration for large teams, faster and easier releases, faster feedback, increased development productivity as a whole, etc.

Preparing an Open-Source Web App for Continuous Deployment

In this section, you will build a simple hello-world app with Node.js. As such, first, you will need to ensure that you have Node.js installed on your machine before moving ahead.

node --version

Running the command above should make your terminal print something similar to v9.11.1. If you get a message saying that node was not found, please, follow this link to install Node.js and NPM.

Scaffolding a Node.js Web App

Now that you have Node.js properly installed in your machine, you will set up the structure of your Node.js application. For easy setup, you will run npm init -y in a new directory (this will be your project root). This command will generate for you a package.json file for that will have the details of your Node.js app. The package.json file contains basic information about your app coupled with the name of the dependencies used.

To create your project and the package.json file, go to your terminal and execute the following commands:

# create the project root
mkdir node-cd

# move into the project root
cd node-cd

# start it as an NPM project
npm init -y

After running the last command, NPM will generate the package.json file for you. Now, if you open the directory for your app, you will see that the package.json file contains the following contents:

{
  "name": "node-cd",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}

Installing the Node.js Web App Dependencies

After creating the package.json file, you will need to install the dependencies needed to build your project. For this article, you will need only two dependencies: express and body-parser. You can install all these dependencies at once by running this command:

npm install express body-parser --save

Once the installation is complete, you should see a node_modules folder. Additionally, your package.json file will contain the dependencies installed and their versions.

Creating a Web Page on Node.js

During the setup of the app, NPM declared the index.js file as the entry point of the app. Now, you need to create this file. So, still in the project root, run the following command to create it:

touch index.js

Next, you will have to create an HTML file. Usually, it is a good practice to create a directory for your views. As such, you will create the directory and the HTML file by running the following commands from the project root:

# create a directory for your views
mkdir pages

# create your first view
touch pages/index.html

Now, to make your Node.js web app serve the HTML file, you will open the index.js file and set it up like so:

// import dependencies
const express = require('express');
const bodyParser = require('body-parser');
const path = require('path');

// initialise express
const app = express();

// root endpoint
app.get('/', (req, res) => {
    res.sendFile(path.join(__dirname + '/pages/index.html'));
});

// select the port in which your Node.js web app will run
const port = 5000;

// then listen to the selected port
app.listen(port, () => console.log(`Server is running on port ${port}`));

This snippet contains all the server logic required for this app. As you can see, just one endpoint is declared (i.e. the one that loads the index.html page). Also, as defined in the code above, the app will run on port 5000.

Now, to add some content to your index.html file, open this file and insert the following code:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <title>Title Page</title>
    <!-- Bootstrap CSS -->
    <link href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" rel="stylesheet">
    <script src="https://oss.maxcdn.com/libs/html5shiv/3.7.3/html5shiv.js"></script>
    <script src="https://oss.maxcdn.com/libs/respond.js/1.4.2/respond.min.js"></script>
</head>
<body>
  <h1 class="text-center">Hello World</h1>  
  <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script>
  <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js"></script>
</body>
</html>

This is a basic HTML code with Bootstrap (a library that helps to build beautiful web apps) and jQuery (a JavaScript library needed by Bootstrap) referenced via CDN (Content Delivery Network). The web page contains a hello world text in an h1 tag to make the text a heading. You can run your server now with this command:

node index.js

Now, if you visit http://localhost:5000, you should see something like this:

Node.js web app localhost "hello world" starting page

That's it! For the purposes of this article, the current project will be enough. Now, you will need to submit your project to a version control system.

GitHub and Open-Source Web Apps

Earlier in this article, I mentioned the need for a repository to demonstrate Continuous Deployment in action. A repository, in this case, is simply a place where source code is stored. As such, you will need to store your project's source code on a remote (online) repository.

Git is the most popular and one of the most advanced version control systems out there. Therefore, a service that can host a Git repository is what you should choose. There are many available out there, but we will use the most popular of them, GitHub. GitHub is a web service where Git repositories are hosted. Actually, they offer more than this and you can read more about GitHub's features here.

Creating a GitHub Account

If you don’t have an account with GitHub, visit the website and create a new one. If you do have an account, make sure you are logged in to your profile. Creating an account requires a unique username, email with any password of your choice.

Creating a GitHub account - Github.com signup form

After registration, you will be required to verify your account through your email address to gain full access.

GitHub 101: Create an Account and a New Repository

GitHub is free for public repositories (i.e. for open-source projects), which is just what you need for this article. To create a new repository, open your profile, click the plus button, and select New repository. Your profile should look like this:

GitHub profile view and creating a new repository

After click on the New repository option, you will be presented with a form like this:

Creating a new GitHub repository

Fill the form with a name of your repository (e.g. node-js-cd-pipeline) and a description (e.g. "Continuous Deployment") then hit the Create repository button. When GitHub finishes creating your repository, you will be redirected to it. There, you will be able to copy the repository's URL (it must be similar to https://github.com/KingIdee/node-js-cd-pipeline.git). Copy it to your clipboard as you will need it soon.

Note: If you have previously configured you GitHub account with an SSH key, you may be able to copy and use the other URL format. This format is similar to: git@github.com:KingIdee/node-js-cd-pipeline.git.

Pushing your Open-Source Web App to GitHub

Haven setup your Node.js web app and created a GitHub repository for it, it is time to push your source code online. So, from your project root, run the following command:

git init

Note: You will need to install the Git CLI (Command Line Interface) to execute git commands.

This command initializes Git in the project directory by creating a .git folder (on most operating systems, this directory is invisible by default). After initializing Git, you will have to set up your GitHub repository as a remote:

git remote add origin REPO_URL

Make sure you replace REPO_URL with the repository URL you copied earlier.

Sometimes, there are particular files that you need to ignore when pushing a project to your remote repository. For this project (and most Node.js projects), you will need to ignore the node_modules directory. Therefore, create a .gitignore file in the directory like so:

touch .gitignore

Then, open the file and paste this line:

/node_modules

Next, you need to commit the changes made to the project and push to your remote repository. You can do so by running the following commands:

# add changed files
git add -A

# commit them to Git
git commit -m "first commit"

# push them to GitHub
git push -u origin master

The first command (git add) adds all files affected by changes to Git, except the ones explicitly listed in the .gitignore file. Then, the second command (git commit) adds a message to the added/changed files to note what changes occurred and save them in your local Git repository. Lastly, the git push command pushes the local changes made to the project to the remote repository (GitHub in this case) and sets up the local project to track remote changes.

If everything works fine, you should see the source code of your project online when you visit the repository you created.

Pushing local changes to the GitHub remote repository

Now.sh and Open-Source Apps

Now is a Platform as a Service (PaaS) which allows you to deploy your projects to the cloud with ease. With Now, you can easily deploy Node.js applications but you can easily deploy apps developed with other languages (like Python, Java, etc). In this case, however, you will need a Dockerfile.

Now aims to make Continuous Deployments easier for developers. Naturally, deploying websites built with Node.js require a sound knowledge of server configurations and management together with a good command of the terminal. With Now, you can focus more on your app logic and worry less about deployments.

Some of the amazing features of Now include:

  • Free unique URL: for every deployment made, there is a unique URL generated usually in the form <appname>-<random string>.now.sh (e.g helloworld-hddnhdvhsd.now.sh).
  • Process logging: every process from the point of running the command to the point of starting the server for the deployed app is logged on the screen and can be viewed by clicking on any of the deployment instance link found on your dashboard.
  • SSL certificate management - Now uses Let's Encrypt to provide your deployments with SSL at no cost. Etc.

What is even more awesome is that Now allows you to deploy lightweight open-source applications on their infrastructure without paying a dime. The code, in that case, is open for anyone who is curious about the project. However, since you are developing an open-source app, this shouldn't be a problem. Also, environment variables (for example credentials to connect to a database) are not shared with the public as they are not part of the source code itself.

Creating a Now.sh Account

To create a Now.sh account, go to the sign-up page of Now and choose one of the methods available (i.e. through email or through your GitHub account). If you choose to sign up through email, you will get a message with a magic link that you can click to verify your account. After that, the browser tab that you used to sign up will be reloaded with your Now dashboard.

Besides that, if you chose the Signup with GitHub option, a verification mail will be sent to your email address.

After verifying your email, you will be able log in to Now. If you are a new user, after logging in, your dashboard should look like this:

Signing up for a Now.sh account

Obtaining a Now Token

Once the registration process is complete and you are logged in, click on your profile picture on the top-right corner and then click on your email address (it will appear under the Settings option). After that, Now will show your account settings and, below your profile picture, you will see a link to Tokens. Click on it.

In this next screen, you will see a section called Authorized Apps and an input field where you will be able to create a new token. In this field, insert a descriptive name like Continuous Deployment and hit the Enter. After that, Now will generate a new Token for you.

For the time being, leave this page open. Soon, you will need to copy this token.

Configuring Travis CI for Continuous Deployment

Travis CI is a Continuous Integration server that you can use for open-source projects. If needed, you can also use it for your private projects but, for this, you will have to use their other website. Continuous Integration servers, as explained before, are used to monitor repositories for changes and to trigger a pre-configured process whenever they find one.

Travis CI particularly supports only projects that are available on GitHub. These projects are usually configured by adding a pipeline file called .travis.yml to the root directory of the repository.

Creating a Travis Account

To move ahead, you have to signup on Travis CI. Travis CI requires only GitHub signup. Since you already have a GitHub account, open the website and signup.

Configuring Travis CI for your GitHub Open-Source Project

After a successful signup, you will be able to see a list of your public repositories on your profile. In there, you will be able to see your repository and you will be able to click on the checkbox button next to it to enable Travis CI to get updates.

Activating the GitHub repository on Travis CI

Configuring Travis in your Open-Source Web App

After activating Travis CI, go back to the project, create a file called .travis.yml, and paste the following code on it:

language: node_js
node_js:
- node
cache:
  directories:
  - node_modules
before_deploy:
  - npm install now --no-save
  - now rm node-cd --token $NOW_TOKEN --yes || true
deploy:
- provider: script
  # deploys the application to now.sh
  script: now --public --token $NOW_TOKEN
  skip_cleanup: true
  on:
    master: true

If you analyze the contents of this file carefully, you will see that:

  • The language property defines what is the main technology used in your project (i.e. Node.js).
  • The before_deploy property specifies the commands to be executed before the deployment phase. In this case, you are telling Travis CI that you will need the Now CLI installed and that it must remove any previous deployment (now rm node-cd). You need this last command because you are using Now's free tier.
  • The deploy section defines recipes (commands) used to deploy your project. In this case, you are telling Travis that you want to use now to deploy changes submitted to the master branch (master: true).

Note: If you are wondering why you need || true on the now rm node-cd command, this is the explanation. For the first time you push your code, you won't have any app on Now, so this command will fail. If you don't add || true, Travis CI will stop the build process because the now rm command will fail.

With this in place, open your package.json file and redefine the scripts section as follows:

{
  "scripts": {
    "start": "node index.js"
  }
}

Giving Travis CI a Now Token

Security is important when deploying your application as secret keys may be involved. For instance, in your case, the Now token is a secret key. Setting the keys as environment variables in Travis CI may seem as the run to approach. Unfortunately, this is not entirely secure. To secure your keys properly, you will need to use the Travis CLI (Command Line Interface). To use this, you need to install Ruby and RubyGems on your machine.

If you don't have them yet, you can follow this documentation to achieve that.

After that, you can use gem to install Travis CLI:

gem install travis

With Travis CLI installed in your local environment, you can use it encrypt your Now Token by running the following command in project root directory:

travis encrypt NOW_TOKEN=YOUR_TOKEN --add

Just make sure you replace YOUR_TOKEN in the command above with the token available in your Now dashboard (remember you left the page open?).

This command encrypts your Now Token and adds its encoded version to .travis.yml under the env section. If the above command is successful you should see some configurations appear on .travis.yml file as shown below:

env:
  global:
    secure: DZC4XpjVPoPl4oXKPD2QATP4++vpPUXllQW0XZjUnSp4S/U9zQamciEMPjvwot324CC/nWm8eL5EyY5WIEyhvhbWwtl85GIYJJeSVhJbkOcwX9Z8Z95aE+9ajI0IMNgE9xS0f8jHYqUOSGTDz0aagGrl8ZgA/qI7qL7QZKLgX07e3nh5Zgyjtrgvyukhchtyiuhoetryuiozb4EoUUc8LJAZJPXBcok/qAmuxPQe6vZt5OTmhNPeL0efdRt861dql45A2qHKOGREYm3Ma0aV1IuqeCLrmoJkT5u7oGd+pG+OWh7LlgA1bjFbTufT/2YiGqCKDNLwbsX8OzBqOlu0Snm8Rb32Yr6VJIw/ulVweg+ZRsEIdNaY=

Travis CI (the only party that will be able to read and decode this property) will use this key to set up the NOW_TOKEN environment variable when running the deployment script. Then, in the end, it will use this variable to deploy your project to Now (through the now --public --token $NOW_TOKEN command).

Defining a Name on Now

The last thing you will have to do to set up Now and Travis together is to create a file called now.json on the project root and add the following contents to it:

{
  "name": "node-cd"
}

This file will tell Now what is the name of your project and it will help Now defining the unique URL when deploying it.

The Continuous Deployment Pipeline in Action

To test your deployment pipeline, all you need to do is add (or change/remove) files, commit, and push to your remote GitHub repository. As you have enabled Travis CI on this repository, GitHub will warn Travis CI about the new changes and the deployment process will start.

Pushing Your First Change

Since you haven't committed and you haven't submitted the changes to .travis.yml, package.json, and now.json yet, you can issue the following commands from your project root directory to push your changes to GitHub:

git add -A

git commit -m "configured the Continuous Deployment pipeline with Travis CI and GitHub"

git push

Once you push the changes to your GitHub repository, Travis CI will trigger a deployment process and take over the build process. As the process goes on, you should see the log on your Travis dashboard.

Travis CI logs while deploying your open-source web app

Note: To find the current job, go to the main Travis CI page while logged in. From there, you will be able to find all running jobs.

Once Travis CI completes the build process, you can visit your Now dashboard to see the current deployment instance. There, you will see, under the Events section, a link to your new deployment.

Now.sh dashboard events after the first deployment

Clicking on it will open your open-source, Node.js app in a new browser tab.

Pushing Your Second Change

Now, to confirm that your Continuous Deployment pipeline truly words, you can make more changes to your pages/index.html file and push to GitHub. So, for example, open this file and replace the h1 element with this:

<h1 class="text-center">
  I've just finished configuring a Continuous Deployment pipeline for my open-source, Node.js web app.
</h1>

Then, go to your terminal and issue the following commands:

git add -A

git commit -m "new message on the index.html file"

git push

Once again, after the last command, GitHub will notify Travis CI about the new changes and a new deployment process will start. After a couple of minutes, Travis CI will finish the build and will use Now to deploy the new version of your open-source Node.js web app.

To see it, just go back to your Now dashboard and click on the icon to open the latest URL defined by Now.

The second deployment of the open-source web app

"I just finished configuring a Continuous Deployment pipeline for my open-source web app."

Aside: Securing Node.js Applications with Auth0

Securing Node.js applications with Auth0 is easy and brings a lot of great features to the table. With Auth0, we only have to write a few lines of code to get solid identity management solution, single sign-on, support for social identity providers (like Facebook, GitHub, Twitter, etc.), and support for enterprise identity providers (like Active Directory, LDAP, SAML, custom, etc.).

In the following sections, we are going to learn how to use Auth0 to secure Node.js APIs written with Express.

Creating the Express API

Let's start by defining our Node.js API. With Express and Node.js we can do this in two simple steps. The first one is to use NPM to install three dependencies: npm i express body-parser cors. The second one is to create a Node.js script with the following code:

// importing dependencies
const express = require('express');
const bodyParser = require('body-parser');
const cors = require('cors');

// configuring Express
const app = express();
app.use(bodyParser.json());
app.use(cors());

// defining contacts array and endpoints to manipulate it
const contacts = [
    { name: 'Bruno Krebs', phone: '+555133334444' },
    { name: 'John Doe', phone: '+191843243223' }
];
app.get('/contacts', (req, res) => res.send(contacts));
app.post('/contacts', (req, res) => {
    contacts.push(req.body);
    res.send();
});

// starting Express
app.listen(3000, () => console.log('Example app listening on port 3000!'));

The code above creates the Express application and adds two middleware to it: body-parser to parse JSON requests, and cors to signal that the app accepts requests from any origin. The app also registers two endpoints on Express to deal with POST and GET requests. Both endpoints use the contacts array as some sort of in-memory database.

We can run and test our application by issuing node index in the project root and then by submitting requests to it. For example, with cURL, we can send a GET request by issuing curl localhost:3000/contacts. This command will output the items in the contacts array.

Registering the API at Auth0

After creating our application, we can focus on securing it. Let's start by registering an API on Auth0 to represent our app. To do this, let's head to the API section of our management dashboard (we can create a free account) if needed) and click on "Create API". On the dialog that appears, we can name our API as "Contacts API" (the name isn't really important) and identify it as https://contacts.mycompany.com/ (we will use this value later).

After creating it, we have to go to the "Scopes" tab of the API and define the desired scopes. For this sample, we will define two scopes: read:contacts and add:contacts. They will represent two different operations (read and add) over the same entity (contacts).

Defining OAuth scopes in the new Auth0 API

Securing Express with Auth0

Now that we have registered the API in our Auth0 account, let's secure the Express API with Auth0. Let's start by installing three dependencies with NPM: npm i express-jwt jwks-rsa express-jwt-authz. Then, let's create a file called auth0.js and use these dependencies:

const jwt = require('express-jwt');
const jwksRsa = require('jwks-rsa');
const jwtAuthz = require('express-jwt-authz');

const tokenGuard = jwt({
  // Fetch the signing key based on the KID in the header and
  // the singing keys provided by the JWKS endpoint.
  secret: jwksRsa.expressJwtSecret({
    cache: true,
    rateLimit: true,
    jwksUri: `https://${process.env.AUTH0_DOMAIN}/.well-known/jwks.json`
  }),

  // Validate the audience and the issuer.
  audience: process.env.AUTH0_AUDIENCE,
  issuer: `https://${process.env.AUTH0_DOMAIN}/`,
  algorithms: ['RS256']
});

module.exports = function (scopes) {
  const scopesGuard = jwtAuthz(scopes || []);
  return function mid(req, res, next) {
    tokenGuard(req, res, (err) => {
      err ? res.status(500).send(err) : scopesGuard(req, res, next);
    });
  }
};

The goal of this script is to export an Express middleware that guarantees that requests have an access_token issued by a trust-worthy party, in this case Auth0. The middleware also accepts an array of scopes. When filtering requests, this middleware will check that these scopes exist in the access_token. Note that this script expects to find two environment variables:

  • AUTH0_AUDIENCE: the identifier of our API (https://contacts.mycompany.com/)
  • AUTH0_DOMAIN: our domain at Auth0 (in my case bk-samples.auth0.com)

We will set these variable soon, but it is important to understand that the domain variable defines how the middleware finds the signing keys.

After creating this middleware, we can update our index.js file to import and use it:

// ... other requires
const auth0 = require('./auth0');

app.get('/contacts', auth0(['read:contacts']), (req, res) => res.send(contacts));
app.post('/contacts', auth0(['add:contacts']), (req, res) => {
  contacts.push(req.body);
  res.send();
});

In this case, we have replaced the previous definition of our endpoints to use the new middleware. We also restricted their access to users that contain the right combination of scopes. That is, to get contacts users must have the read:contacts scope and to create new records they must have the add:contacts scope.

Running the application now is slightly different, as we need to set the environment variables:

export AUTH0_DOMAIN=bk-samples.auth0.com
export AUTH0_AUDIENCE="https://contacts.mycompany.com/"
node index

Let's keep this API running before moving on.

Creating an Auth0 Application

As the focus of this section is to secure Node.js applications with Auth0, we are going to use a live Angular app that has a configurable Auth0 application. Before using this app, we need to create an Auth0 application that represents it. Let's head to the "Applications" section of the management dashboard and click on the "Create Application" button.

On the popup shown, let's set the name of this new application as "Contacts Application" and choose "Single Page Web App" as the application type. After hitting the "Create" button, we have to go to the "Settings" tab and set http://auth0.digituz.com.br/callback in the "Allowed Callback URLs" field.

Now we can save the application and head to the sample Angular app secured with Auth0. To use this app, we need to set the correct values for four properties:

  • clientID: We have to copy this value from the "Client ID" field of the "Settings" tab of "Contacts Application".
  • domain: We can also copy this value from the "Settings" tab of "Contacts Application".
  • audience: We have to set this property to meet the identifier of the "Contacts API" that we created earlier.
  • scope: This property will define the authority that the access_token will get access to in the backend API. For example: read:contacts or both read:contacts add:contacts.

Then we can hit the "Sign In with Auth0" button.

Using the Angular app with the configurable Auth0 application

After signing in, we can use the application to submit requests to our secured Node.js API. For example, if we issue a GET request to http://localhost:3000/contacts/, the Angular app will include the access_token in the Authorization header and our API will respond with a list of contacts.

Getting a response from a secure Node.js API

Conclusion

In this article, you had the chance to deal with one of the buzzing terms in modern software development - Continuous Deployments. You also took a brief look into Git, Continuous Integration servers, and their respective duties in Continuous Deployments. Particularly, you used tools like GitHub, Travis CI, and Now.sh to configure a Continuous Deployment pipeline for open-source web apps. Isn’t that awesome?

With this knowledge, you can go ahead and apply Continuous Integration to your much more complex projects. You can even decide to try tools similar to what is used here, like trying different CI servers, or a new Git hosting web service.

I look forward to seeing what you will build. Cheers!