Sometimes users will apply theÂ @NonCPS   annotation to a method definition in order to bypass the CPS transform inside that method. This can be done to work around limitations in Groovy language coverage (since the body of the method will execute using the native Groovy semantics), or to get better performance (the interpreter imposes a substantial overhead). However, such methods must not call CPS-transformed code such as Pipeline steps. For example, the following will not work:

Using theÂ node   orÂ sh   steps from this method is illegal, and the behavior will be anomalous. The warning in the logs from running this script looks like this:

To fix this case, simply remove the annotation — it was not needed. (Longtime Pipeline users might have thought it was, prior to the fix of JENKINS-26481.)

Some Groovy and Java methods take complex types as parameters to support dynamic behavior. A common case is sorting methods that allow callers to specify a method to use for comparing objects (JENKINS-44924). Many similar methods in the Groovy standard library work correctly after the fix for JENKINS-26481, but some methods remain unfixed. For example, the following will not work:

The closure passed toÂ Iterable.toSorted is CPS-transformed, but Iterable.toSorted itself is not CPS-transformed internally, so this will not work as intended. The current behavior is that the return value of the call to toSorted will be the return value of the first call to the closure. In the example, this results inÂ sorted being set to -1 , and the warning in the logs looks like this:

To fix this case, any argument passed to these methods must not be CPS-transformed. This can be accomplished by encapsulating the problematic method ( Iterable.toSorted in the example) inside another method, and annotating the outer method withÂ @NonCPS , or by creating an explicit class definition for the closure and annotating all methods on that class withÂ @NonCPS .

Occasionally, users may attempt to use CPS-transformed code such as Pipeline steps inside of a constructor in a Pipeline script. Unfortunately, the construction of objects via theÂ new  operator in Groovy is not something that can be CPS-transformed (JENKINS-26313), and so this will not work. Here is an example that calls a CPS-transformed method in a constructor:

The construction ofÂ Test  will fail when the constructor calls Test.setX because setX  is a CPS-transformed method. The warning in the logs from running this script looks like this:

To fix this case, ensure that any methods defined in a Pipeline script that are called from inside of a constructor are annotated withÂ @NonCPS  and that constructors do not call any Pipeline steps. If you must call CPS-transformed code such a Pipeline steps from the constructor, you need move the logic related to the CPS-transformed methods out of the constructor, for example into a static factory method that calls the CPS-transformed code and then passes the results to the constructor.

Users may create a class in a Pipeline Script that extends a preexisting class defined outside of the Pipeline script, for example from the Java or Groovy standard libraries. When doing so, the subclass must ensure that any overriding methods are annotated with @NonCPS  and do not use any CPS-transformed code internally. Otherwise, the overriding methods will fail if called from a non-CPS context. For example, the following will not work:

Calling the CPS-transformed override ofÂ toString  from non-CPS-transformed code such as StringBuilder.append is not permitted and will not work as expected in most cases. The warning in the logs from running this script looks like this:

To fix this case, add the @NonCPS annotation to the overriding method, and remove any uses of CPS-transformed code such as Pipeline steps from the method.

In Groovy, it is possible to use a closure in a GString so that the closure is evaluated every time the GString is used as a String . However, in Pipeline scripts, this will not work as expected, because the closure inside of the GString will be CPS-transformed. Here is an example:

Using a closure inside of aÂ GString   as in this example will not work. The warning from the logs when running this script looks like this:

To fix this case, replace the original GString with a closure that returns a GString that uses a normal expression rather than a closure, and then call the closure where you would have used the original GString  as follows:

Below are two examples of the Pipeline Linter in action. This first example shows the output of the linter when it is passed an invalid Jenkinsfile , one that is missing part of the agent declaration.

In this second example, the Jenkinsfile has been updated to include the missing any on agent . The linter now reports that the Pipeline is valid.

To use the "Replay" feature:

Once you are satisfied with the changes, you can use Replay to view them again, copy them back to your Pipeline job or Jenkinsfile , and then commit them using your usual engineering processes.

The Jenkins Editor Eclipse plugin can be found on Eclipse Marketplace. This special text editor provides some features for defining pipelines e.g:

The Jenkins Pipeline Linter Connector extension for VisualStudio Code takes the file that you have currently opened, pushes it to your Jenkins Server and displays the validation result in VS Code.

​You can find the extension from within the VS Code extension browser or at the following url: marketplace.visualstudio.com/items?itemName=janjoerke.jenkins-pipeline-linter-connector

The extension adds four settings entries to VS Code which select the Jenkins server you want to use for validation.

The nvim-jenkinsfile-linter Neovim plugin allows you to validate a Jenkinsfile by using the Pipeline Linter API of your Jenkins instance and report any existing diagnostics in your editor.

The linter-jenkins Atom package allows you to validate a Jenkins file by using the Pipeline Linter API of a running Jenkins. You can install it directly from the Atom package manager. It needs also to install Jenkinsfile language support in Atom

The Jenkinsfile Sublime Text package allows you to validate a Jenkinsfile by using the Pipeline Linter API of a running Jenkins instance over a secure channel (SSH). You can install it directly from the Sublime Text package manager.

​You can find the package from within the Sublime Text interface via the Package Control package, at GitHub, or packagecontrol.io:

Short: if it is important to keep workspace synchronized with other stages, use reuseNode true . Otherwise, dockerized stage can be run on any other agent or on the same agent, but in temporary workspace.

By default, for containerized stage, Jenkins does:

If you have multiple Jenkins agents, your containerized stage can be started on any of them.

When reuseNode set to true : no new workspace will be created, and current workspace from current agent will be mounted into container, and container will be started at the same node, so whole data will be synchronized.

Many build tools will download external dependencies and cache them locally for future re-use. Since containers are initially created with "clean" file systems, this can result in slower Pipelines, as they may not take advantage of on-disk caches between subsequent Pipeline runs.

Pipeline supports adding custom arguments which are passed to Docker, allowing users to specify custom Docker Volumes to mount, which can be used for caching data on the agent between Pipeline runs. The following example will cache ~/.m2 between Pipeline runs utilizing the maven container, thereby avoiding the need to re-download dependencies for subsequent runs of the Pipeline.

It has become increasingly common for code bases to rely on multiple, different, technologies. For example, a repository might have both a Java-based back-end API implementation and a JavaScript-based front-end implementation. Combining Docker and Pipeline allows a Jenkinsfile to use multiple types of technologies by combining the agent {} directive, with different stages.

For projects which require a more customized execution environment, Pipeline also supports building and running a container from a Dockerfile in the source repository. In contrast to the previous approach of using an "off-the-shelf" container, using the agent { dockerfile true } syntax will build a new image from a Dockerfile rather than pulling one from Docker Hub.

Re-using an example from above, with a more custom Dockerfile :

By committing this to the root of the source repository, the Jenkinsfile can be changed to build a container based on this Dockerfile and then run the defined steps using that container:

The agent { dockerfile true } syntax supports a number of other options which are described in more detail in the Pipeline Syntax section.

By default, Pipeline assumes that any configured agent is capable of running Docker-based Pipelines. For Jenkins environments which have macOS, Windows, or other agents, which are unable to run the Docker daemon, this default setting may be problematic. Pipeline provides a global option in the Manage Jenkins page, and on the Folder level, for specifying which agents (by Label) to use for running Docker-based Pipelines.

The /usr/local/bin directory is not included in the macOS PATH for Docker images by default. If executables from /usr/local/bin need to be called from within Jenkins, then the PATH needs to be extended to include /usr/local/bin . Add a path node in the file "/usr/local/Cellar/jenkins-lts/XXX/homebrew.mxcl.jenkins-lts.plist" like this:

The revised PATH string should be a colon separated list of directories in the same format as the PATH environment variable and should include:

Now restart jenkins using "brew services restart jenkins-lts"

Using Docker in Pipeline can be an effective way to run a service on which the build, or a set of tests, may rely. Similar to the sidecar pattern , Docker Pipeline can run one container "in the background", while performing work in another. Utilizing this sidecar approach, a Pipeline can have a "clean" container provisioned for each Pipeline run.

Consider a hypothetical integration test suite which relies on a local MySQL database to be running. Using the withRun method, implemented in the Docker Pipeline plugin’s support for Scripted Pipeline, a Jenkinsfile can run MySQL as a sidecar:

This example can be taken further, utilizing two containers simultaneously. One "sidecar" running MySQL, and another providing the execution environment, by using the Docker container links.

The above example uses the object exposed by withRun , which has the running container’s ID available via the id property. Using the container’s ID, the Pipeline can create a link by passing custom Docker arguments to the inside() method.

The id property can also be useful for inspecting logs from a running Docker container before the Pipeline exits:

In order to create a Docker image, the Docker Pipeline plugin also provides a build() method for creating a new image, from a Dockerfile in the repository, during a Pipeline run.

One major benefit of using the syntax docker.build("my-image-name") is that a Scripted Pipeline can use the return value for subsequent Docker Pipeline calls, for example:

The return value can also be used to publish the Docker image to Docker Hub, or a custom Registry, via the push() method, for example:

One common usage of image "tags" is to specify a latest tag for the most recently, validated, version of a Docker image. The push() method accepts an optional tag parameter, allowing the Pipeline to push the customImage with different tags, for example:

The build() method builds the Dockerfile in the current directory by default. This can be overridden by providing a directory path containing a Dockerfile as the second argument of the build() method, for example:

It is possible to pass other arguments to docker build by adding them to the second argument of the build() method. When passing arguments this way, the last value in the that string must be the path to the docker file and should end with the folder to use as the build context)

This example overrides the default Dockerfile by passing the -f flag:

By default, the Docker Pipeline plugin will communicate with a local Docker daemon, typically accessed through /var/run/docker.sock .

To select a non-default Docker server, such as with Docker Swarm, the withServer() method should be used.

By passing a URI, and optionally the Credentials ID of a Docker Server Certificate Authentication pre-configured in Jenkins, to the method with:

By default the Docker Pipeline integrates assumes the default Docker Registry of Docker Hub.

In order to use a custom Docker Registry, users of Scripted Pipeline can wrap steps with the withRegistry() method, passing in the custom Registry URL, for example:

For a Docker Registry which requires authentication, add a "Username/Password" Credentials item from the Jenkins home page and use the Credentials ID as a second argument to withRegistry() :

If you are new to Jenkins Pipeline, the Blue Ocean UI helps you set up your Pipeline project, and automatically creates and writes your Pipeline (i.e. the Jenkinsfile ) for you through the graphical Pipeline editor.

As part of setting up your Pipeline project in Blue Ocean, Jenkins configures a secure and appropriately authenticated connection to your project’s source control repository. Therefore, any changes you make to the Jenkinsfile via Blue Ocean’s Pipeline editor are automatically saved and committed to source control.

Read more about Blue Ocean in the Blue Ocean chapter and Getting started with Blue Ocean page.

A Jenkinsfile created using the classic UI is stored by Jenkins itself (within the Jenkins home directory).

To create a basic Pipeline through the Jenkins classic UI:

Complex Pipelines are difficult to write and maintain within the classic UI’s Script text area of the Pipeline configuration page.

To make this easier, your Pipeline’s Jenkinsfile can be written in a text editor or integrated development environment (IDE) and committed to source control ^[3] (optionally with the application code that Jenkins will build). Jenkins can then check out your Jenkinsfile from source control as part of your Pipeline project’s build process and then proceed to execute your Pipeline.

To configure your Pipeline project to use a Jenkinsfile from source control:

When you update the designated repository, a new build is triggered, as long as the Pipeline is configured with an SCM polling trigger.

The built-in "Snippet Generator" utility is helpful for creating bits of code for individual steps, discovering new steps provided by plugins, or experimenting with different parameters for a particular step.

The Snippet Generator is dynamically populated with a list of the steps available to the Jenkins instance. The number of steps available is dependent on the plugins installed which explicitly expose steps for use in Pipeline.

To generate a step snippet with the Snippet Generator:

To access additional information and/or documentation about the step selected, click on the help icon (indicated by the red arrow in the image above).

In addition to the Snippet Generator, which only surfaces steps, Pipeline also provides a built-in " Global Variable Reference ." Like the Snippet Generator, it is also dynamically populated by plugins. Unlike the Snippet Generator however, the Global Variable Reference only contains documentation for variables provided by Pipeline or plugins, which are available for Pipelines.

The variables provided by default in Pipeline are:

This video reviews using the currentBuild variable in Jenkins Pipeline.

While the Snippet Generator helps with generating steps for a Scripted Pipeline or for the steps block in a stage in a Declarative Pipeline, it does not cover the sections and directives used to define a Declarative Pipeline. The "Declarative Directive Generator" utility helps with that. Similar to the Snippet Generator, the Directive Generator allows you to choose a Declarative directive, configure it in a form, and generate the configuration for that directive, which you can then use in your Declarative Pipeline.

To generate a Declarative directive using the Declarative Directive Generator:

The Directive Generator can generate configuration for nested directives, such as conditions inside a when directive, but it cannot generate Pipeline steps. For the contents of directives which contain steps, such as steps inside a stage or conditions like always or failure inside post , the Directive Generator adds a placeholder comment instead. You will still need to add steps to your Pipeline by hand.

A Jenkinsfile can be written using two types of syntax - Declarative and Scripted.

Declarative and Scripted Pipelines are constructed fundamentally differently. Declarative Pipeline is a more recent feature of Jenkins Pipeline which:

Many of the individual syntactical components (or "steps") written into a Jenkinsfile , however, are common to both Declarative and Scripted Pipeline. Read more about how these two types of syntax differ in Pipeline concepts and Pipeline syntax overview below.

A Pipeline is a user-defined model of a CD pipeline. A Pipeline’s code defines your entire build process, which typically includes stages for building an application, testing it and then delivering it.

Also, a pipeline block is a key part of Declarative Pipeline syntax.

A node is a machine which is part of the Jenkins environment and is capable of executing a Pipeline.

Also, a node block is a key part of Scripted Pipeline syntax.

A stage block defines a conceptually distinct subset of tasks performed through the entire Pipeline (e.g. "Build", "Test" and "Deploy" stages), which is used by many plugins to visualize or present Jenkins Pipeline status/progress. ^[6]

A single task. Fundamentally, a step tells Jenkins what to do at a particular point in time (or "step" in the process). For example, to execute the shell command make use the sh step: sh 'make' . When a plugin extends the Pipeline DSL, ^[1] that typically means the plugin has implemented a new step .

In Declarative Pipeline syntax, the pipeline block defines all the work done throughout your entire Pipeline.

In Scripted Pipeline syntax, one or more node blocks do the core work throughout the entire Pipeline. Although this is not a mandatory requirement of Scripted Pipeline syntax, confining your Pipeline’s work inside of a node block does two things: