A while ago I wrote a post about continuous integration with Jenkins for scala projects and I thought I would do one for node developers also. Make sure to check out my other posts about how to write your gulp files since we’ll heavily really on those gulp tasks. I will put the links at the end of this post. So what is continuous integration, at the very high level it’s about hey is the code in a good state, no matter by how many people code is being updated, how often code is being changed or of how many building blocks project consists of. When you start working on a feature you’re gonna branch of mainline and start introducing changes. At some point when you find things are broken now you have to spend time to see if it’s because of your changes or project was already in a bad state. And when you find out that it was already in a bad state you would want to start tracing. A who introduced the breaking changes and B when. So to avoid the head each and frustration what you really want to know is if project is in a good state before you even start working on it. One option would be to compile the code before you start changing it. But if code compiles is it enough? Oh you you also wanna know if unit tests are passing. But is this enough? Check for compiler warnings? Style violations? Code coverage? What if you run code coverage and it shows that it’s 85%, does it mean everything is good, what if code coverage was 95% last week and code without proper code coverage got committed and code coverage dropped (do you remember what code coverage was last week, can you even find out without going back in commits and running coverage reports on those commits.) are you sure you want to introduce your changes on top of those changes? Let’s make it even more complex, what if the project consists of many other projects and the project you’re working on brings all those dependencies tougher. Maybe you should checkout all those dependency projects and see if those are in a good state also. Let’s make it even more complex, what if the project isn’t just set of projects that get bundled into one big project. What if the whole code base consists of many micro services that talk to each other via some form of PRC. OK maybe you can manage to do all this to know if code is in a good state before you introduce your changes what about the point B I brought earlier about when code broke, because longer it was in that states longer it’s gonna take to trace the breaking change and longer to fix. Can you go and check every single project every time someone makes a commit to make sure that commit is good? Well actually you can, you just need a build server. So all those points I was bringing was about whatever your integration is it should be continuous and it’s continuous if it runs after somebody changes something (or maybe before is a better option?☺ I’ll get to this later.)
This should cover what the continuous part is but what about integration, what is it actually? I would define it as pipeline of checks that run one after another and if one fails the whole thing fails if all pass then code is good. Now let’s cover what checks you want to do. You’re the best person to know what checks you need and what you wanna track. I will cover the most common ones, at least the ones that are relevant for Node projects. Also I would cover how to do it in Jenkins since Jenkins is my preferred build server but it should me mostly similar other the other ones.
Before getting to what pipeline should be, let’s figure out what the node environment is gonna look like. One intuitive thing to do would be to install node, npm and all global packages in Jenkins box, provision it with Chef or even better, have a Jenkins docker image. There are few problems with this approach though. One is that every time you need a new global package you need to re-provision Jenkins instance or re-build the docker image. Other big problem is the versioning issue, one project may need one version of node or global packages and another project may need other version, we don’t want to be forced to upgrade all the project to latest versions in one day. What would be ideal is isolated node environments. Luckily Jenkins has NodeJS plugin. What it does it installs isolated node environments in Jenkins’s home directory. After installing it in Configure System add NodeJs Installation specify the version of node to use, global packages to install and give it a name.
Later in job configuration, under Build Environment section, select Provide Node & npm bin/ folder to PATH and pick the node installation you wanna use for that project.
After this, we can provide the commands to execute for our build. This is how it’ll look like for library projects.
Let’s dig deep to understand what we’re doing here. First we do npm install, so if a commit has a new dependency or upgraded a dependency we’ll have it installed. Next we run gulp build, which does all the heavy lifting. It will lint the code, test it, generate coverage report, and compile CoffeeScript to JavaScript. I’m passing JUNIT_REPORT_PATH and JUNIT_REPORT_STACK environment variables to generate jUnit xml file, which we’ll use little later to generate graphs and reports. Next I run gulp lint task on it’s own, setting JENKINS environment variable so my gulp script knows that it’s running in Jenkins and it needs to generate checkstyle xml. The reason I’m running it separately and piping it to a file is because the gulp plugin I’m using for CoffeeScript linting outputs checkstyle xml on the console and the easiest way to get it into an xml file and let Jenkins’s checkstyle plugin know about it is to pipe it to a file. After the build succeeds I run npm publish to publish my npm and run gulp version, store it in build.properties which I’m gonna inject into Jenkins environment and tag the git commit with that version later.
At this point build is done. What we have is report files that we’re gonna use to generate graphs and reports. We also published the npm so let’s also tag the commit with same version. And at the end send slack and email notifications.
Checkstyle
In our build we ran gulp lint and stored the output in build/checkstyle-report-coffee.xml so let’s point checkstyle plugin to it:
What we’ll get is nice report pages where we can drill down to see what the warnings and errors are, in which files, line numbers and event see the line of code. We’ll get nice summery on job page, also trend graph.
Open tasks
Another thing that would be helpful is to know if there are open tasks in project and if there are, what those are. Jenkins has a way to detect tags like TODO, FIXME and others. It scans the code to find those, let’s configure it. Notice that I’m excluding node_modules to not count open tasks in dependency modules.
What we’ll get is a nice reports and trend graph on our job page.
Unit tests
In our build script we put JUNIT_REPORT_PATH=build/test-report.xml so our jUnit xml reports file are in that file, let’s just point jenkins to it.
What we’ll get is a nice reports and trend graph on our job page.
Code coverage
When unit testes are passing it’s still not enough we want to measure what the code coverage is and of course find code paths that are never being hit from a unit tests. If you look in my gulp files you’ll see that it generates cobertura reports also (in build/coverage/cobertura-coverage.xml file.) So let’s use it.
What we’ll get is a nice reports and trend graph on our job page.
Output of the build
We define build to be a pipeline of checks that the code goes through every time there is a commit. We torture every single commit to go through this pipeline and we say if build passes commit is good. So what is the output of the build? Also we only deploy good changes, right? The output of a good build would be the artifacts. We worked really hard to make sure that code is good by running series of checks on it and we better keep those artifacts for deployment. Since this one project is a library project the artifact is the npm package and I’m publishing it to my npm repository (in may instance nexus, you may use npmjs.) In case of server apps artifact can be a docker image job will build and push to docker registry. For other type of projects zip file can be good enough, just version it and put on S3 or something.
Versioning
And since every good commit produces an artifact (in this case npm package is the artifact) we should talk about versions. One thing that would be good if we could link a specific version of an artifact to source code. A good way to do this is to create a tag on commit when we build the artifact. We already have the version in package.json, I defined version gulp task to get it and was doing some serious gymnastics in build script to get it into VERSION environment variable. I would just use Git publisher functionality of Jenkins to create a git tag with that version.
Notifications
So build pipeline is there, now every time there is a commit in source control build will trigger and it will either fail or pass. Either way we get instant feedback which was one of the main problems we wanted to solve. You can get notifications when job stars, fails or succeeds. I generally don’t care to be notified when a job succeeds unless it’s deploy job or something (which I won’t be covering in this post.) but I absolutely care to know when a build job is failing. You can do pretty much all types of notifications: Email, Slack, Hipchat, IRC, IM whatever you’re using, Jenkins has plugins for all of those. I personally like to send an email out when a job fails and put a message in a Slack room when a job starts or finishes.
And include Slack notifications at post build step at the bottom of job configuration.
Pull requests
Remember I said it would be better to know if a commit is good before it even gets to mainline. With the setup mentioned so far when a commit gets to master build will trigger and if it’s bad job will fail. To not allow bad code to even get to master at first place we need to run integration on it before merging it. We never merge a branch when you do code review and identify that it needs more work, right? But what you can’t tell from just code reviewing is if it compiles, unit tests are passing, what is code coverage after the change, etc. And do we even want to code review a change if build will fail for it? What we need to do is to have a magical way that when we go to review a pull request somehow it has a flag on it saying build is passing for it or no. There is a Jenkins plugin for building pull requests also, it will run the build, and comment github pull request.
To get this working all you need to do is to create another job that instead of building master will build pull requests. For this job there are few things we don’t actually need. We don’t care do generate scala docs for this type of builds, version or publish the artifacts, so make sure to exclude those steps from pull request builder jobs.
Wrapping up
Let’s see what we got. Every time there is a commit build will start immediately and we’ll get instant feedback whether build passes or fails. If build passes we know that all checks we defined are good for the commit. We create tags for each version in git, we publish artifacts for deployment or referencing from other projects. Other than those we get lot of nice features on Jenkins dashboard.
- We can see the status of the build.
- See the change log with commit messages, committers and links to github.
- We can browser the workspace.
- Start a build.
- Navigate to github page of the project.
- See detailed reports for checkstyle or other static analysis warnings.
- Navigate to scala docs.
- See code full coverage reports with percentages, which lines are covered which ones aren’t, classes, packages.
Make sure to also check out my other posts about gulp files and how to run node apps in docker: