You shall not pass – Control your code quality gates with a wizard – Part I

You shall not pass – Control your code quality gates with a wizard

Now as easy as designing an email filter

Every project has different policies defining when code can be pushed into production. CollabNet’s code quality gate wizard for Gerrit comes with a bunch of predefined policies and lets you graphically design your own quality gates as easy as defining email filter rules.

Four-eye peer review, legal has to approve copyright file changes, senior staff has to approve the work of juniors, democratic feature voting? – Regardless of what your code quality gates look like, chances are very high you can now enforce it without having to write a single line of code.

What are Quality Gates – And Why should I care?

Quality Gates applied before commits get directly pushed into production

The days where version control was just a passive component where you store your source code are long gone. In the world of continuous integration and delivery, version control is an active component, typically the first step in a (semi) automated pipeline straight through production. This pipeline is typically automatically triggered whenever a commit gets merged into master (or any other branch used for production). If a commit that does not meet audit compliance or production quality gets merged accidentally, this can have immediate effect on your business. In the worst case, you face data loss, customers cannot interact with your business anymore or you are getting sued for having introduced a security hole or serious malfunction.

Code quality gates define the conditions to be met before a commit can be merged into master, i.e. when code is ready to be pushed for production. Typically those conditions are a mixture of automated checks like passing unit and integration tests, code quality and guideline checkers as well human checks like peer review, approval from legal and product management.

Having those rules automatically enforced is a big win for every team as it will make sure you always have the quality level and compliance conformance you need in production.

With CollabNet’s new quality gate wizard for Gerrit – TeamForge’s Git backend – you can now select from a number of predefined policies (best practices quality gates) which will be automatically enforced once deployed. In addition, you can design your own quality gates without having to write a single line of code. The way it works is very similar to email filter rules: You define the characteristics of a commit and related context (like associated peer reviews and feedback from code quality tools, system and integration tests) and decide whether under those conditions the commit can go in or not. You can even point to already existing commits and their context to automatically create quality gates and simulate them within the wizard.

This blog post series consists of three blogs. In the first one (the one you are reading right now), you will learn how to install the quality gate wizard and how to deploy the out of the box policies (collection of best practice quality gates), that come with the wizard.

In the second blog post, you will learn how to design your own quality gates based on the email filter metaphor. Furthermore, you will get an answer on how to define and distribute your own best practice policies for your organization.

The third blog post gets pretty technical and will dive into the more advanced concepts of the wizard, like defining filters on commit characteristics, counting peer review and CI votes. It will also explain the specifics of the language that is generated by the wizard to implement the quality gates.

With that said, let’s jump right in.

Make sure your Git/Gerrit Backend supports Quality Gates

If you are using TeamForge with our Git/Gerrit integration version 8.2.0 or higher, the quality gate backend is already installed. Otherwise, you would have to upgrade to this version which is supported by both TeamForge 7.1 as well as TeamForge 7.2. More details can be found on http://help.collab.net

Installing the code quality gate wizard

Designing code quality gates is a feature for power users. For that reason, we decided to implement the first version of the wizard inside our CollabNet Desktop for Eclipse and GitEye. If it turns out that you really love this feature and need a Web UI for it, we can make that happen too. As usual, just drop a comment in this blog post for any kind of feedback.

You can install any of the tools mentioned, my colleague Steve wrote two blog posts on how to install GitEye and how to set it up with TeamForge and Gerrit. If you already have GitEye or any other Eclipse based application installed and want to add the Quality Gate Wizard, point Eclipse to our update site http://downloads.open.collab.net/eclipse/update-site/gerrit-workflow and install all plugins available there.

Opening the quality gate wizard and selecting a predefined policy

Once you have installed GitEye or CollabNet’s Desktop for Eclipse and configured your TeamForge site, let’s navigate to the Git repository where you want to deploy some quality gates. Right click on the repository of your choice (in our case TeamForge-Git-Integration) and select the option Define Gerrit Review Rules …

A screen similar to the one depicted below will open. Within that screen, the option Load from template is already pre-selected. It contains a number of predefined policies (collection of best practice quality gates). The one we are using is called Relaxed 4 Eye Principle and 1+1.

You can skip over the details of this policy now, but if you are interested, here are the quality gates enforced:

• Every commit has to be verified by a continuous integration (CI) system like Jenkins. The job of this system is to ensure that the code compiles, unit, integration and system tests are running through fine and all coding guidelines and code quality metrics are satisfied.

• Every commit has to be peer reviewed by at least one person other than the author of the commit (4 Eye-Principle)

• If a peer reviewer vetoes the commit, it cannot go in

• If at least one reviewer strongly approves the commit (Code-Review +2) or at least two reviewers agree that the commit has reasonable quality (sum of Code-Review votes >=2), the commit can be merged if all conditions above are satisfied

We chose this policy as an example as this is the one we are internally following while developing our TeamForge-Git-Integration.

Testing the policy in the wizard

Once you click Finish in the wizard, an editor will open within Eclipse. We will cover most of its functionality in subsequent blog posts. For now, all we need to know are two buttons: Test Against Gerrit Change and Deploy to Gerrit. With the first one, you can test your quality gates against any commit not yet merged into a branch (to be more technically precise, any Gerrit change). The screenshot below shows how the current selection of quality gates would react to a particular commit. In the case below, the continuous integration system which tried to build the commit, ran unit and integration tests and checked code quality metrics, voted against the commit, so it cannot be pushed into production in its current form (red traffic lights). The yellow traffic lights indicate that no quality gate vetoed against the particular commits but there are still elements missing in order to let it pass (CI feedback or peer review feedback from a non-author). One commit (associated with Gerrit change 1985) has a green traffic light and could be pushed into production if needed.

Deploying the policy 

Once you are satisfied with your code quality gates, you can deploy them, i.e. make sure they are enforced for any commit in the repository in question. To do that, just hit the Deploy to Gerrit button (you need SCM admin permissions in TeamForge to make this work). A wizard will open that lets you enter your credentials for the Git repository in question and lets you specify a message that goes with your quality gate deployment (behind the scenes, quality gates are versioned in Gerrit as Git commits, so you can see any change to your policies and even revert back if needed).

Checking the result in Gerrit

If you now log into Gerrit’s Web UI (or use the Gerrit Mylyn Plugin), you can see the quality gates in action. In the screenshot below, you can see that

• the commit in question has been already verified (green verified checkbox)

• the commit in question has been already strongly recommended by a reviewer (green checkbox in Code-Review)

However, as the commit in question has been authored by the reviewer himself (see owner and reviewer fields), it cannot go into production yet. At the bottom of the screenshot, you see a message indicating that a Non-Author has to do Code-Review.

Summary

In this blog post you learned how to select, test and deploy predefined quality gates with CollabNet’s code quality gate wizard for Gerrit. Those quality gates will make sure that all conditions regarding code quality and compliance are met before a commit can be merged into your master branch and trigger a pipeline that will eventually promote it into production.

In the next blog posts we will focus on how you can define quality gates by yourself, using a methodology very close to setting up email filter rules.