Jenkins-as-code: job-dsl-plugin

In the first post in this series, I covered the problems we were having with job creation and maintenance and a very high level look at our solution.

In this post, I’ll go more in depth with the solution, covering the job-dsl-plugin. I’m not going to recreate others’ documentation here, so for actual instructions, I’ll simply link to existing docs.

job-dsl-plugin

I’m assuming if you’re reading this that you’re quite familiar with creating Jenkins jobs, point-and-click style, via the Jenkins GUI.  job-dsl-plugin is a Jenkins plugin that enables you to define Jenkins jobs in plain text, using a really nice Groovy-based domain specific language (DSL).

Basic example

There’s no limit to the number of jobs you can keep in a single script.

There’s no limit to the number of scripts you can have.

At my organization, we’ve observed that some teams like to keep everything in a single “jobs.groovy”; some teams like a single job per script; some split jobs across functional lines, such as “Build Flows” and “Management Tasks”. job-dsl-plugin imposes no limits on how you structure your jobs with respect to file system. The only requirement is that the script files be valid groovy script names, and the contents of the scripts be valid Groovy.

I’ll go more in depth momentarily, but I imagine the burning question right now is, so how do I turn this DSL into actual Jenkins jobs?

Seed jobs

Your job scripts live in source control, but how do you turn them into Jenkins jobs? The answer: seed jobs.

A seed job is a job you create in Jenkins, pulling in the repo where you keep your job scripts, with a Build Step of Process Job DSLs. You tell that build step where your job scripts live in the workspace.

Then, you run the job. If all is well, it’ll create the jobs as you’ve configured them in your groovy script(s).

It’s a good idea to set up your seed job to run when your jobs SCM repo is updated, obviously, so add that SCM trigger.

For a fuller treatment of seed jobs, follow the tutorial.

How does this whole thing work, anyway?

When you create a job in the Jenkins GUI, that job is stored in config.xml.

The job-dsl-plugin is simply another way of creating config.xml; in this case, it’s by processing a DSL, not by pointing and clicking.

DSL support for plugins is currently added in 3 ways:

1. by contributing to the job-dsl-plugin itself. Many, many Jenkins plugins have already been added, and the plugin is currently very actively maintained and updated frequently.
2. by plugins implementing the job-dsl Extension points
3. the newest method, the auto-generated DSL

This last method does has some potentially serious limitations, namely only running in Jenkins and not via the command line (an important one for us; more on that in a later post).

But what if a plugin hasn’t been added to the DSL API? Enter configure blocks.

Custom configure blocks

When we were investigating text-based job solutions, one requirement was that any solution we chose must not limit us in any way. We needed assurance that anything we could do in the Jenkins GUI, we could do in text.

job-dsl-plugin satisfies this requirement. While all core jenkins job behavior and many plugins are available directly in the DSL — for example, “triggers” and “cron” and “steps” — anything not currently supported in the DSL is fairly easily added to a job via a configure block

These blocks — just like all DSL configurations — ultimately result in modifications to the job’s config.xml file. The configure block is a direct mapping of keywords to XML elements and attributes:

Will result in the job’s config.xml having a new element ‘com.checkmarx.jenkins.CxScanBuilder’ added under the project’s builders, with sub-elements for serverUrl, username, password, and so forth. The doc, linked above, clearly illustrates how to use custom configure blocks.

There is an up-front cost to using configure blocks: first, you have to find or create a job and configure it in the Jenkins UI, then inspect the generated XML to see what elements and attributes you need to use in the configure block. For us, in the handful of cases we’ve needed to use custom configure blocks, it’s well worth the few minutes of manual configuration to learn what we need to learn. Plus, configure blocks are easily reusable.

Learning more

The job-dsl API viewer is pure gold.

As is the job-dsl wiki

Next up: our custom builders

Now that I’ve covered the job-dsl foundation, in the next post I’ll describe our approach to creating reusable job builders for the patterns we saw emerge across projects.