Welcome to Knowledge Base!

KB at your finger tips

This is one stop global knowledge base where you can learn about all the products, solutions and support features.

Categories
All

DevOps-Jenkins

Managing Nodes

Managing Nodes

Table of Contents
  • Components of Distributed Builds
  • Creating Agents
  • Launch inbound agent via Windows Scheduler
  • Installing a Jenkins agent on Windows
  • Creating a macOS agent for Jenkins

Components of Distributed Builds

Builds in a Distributed Builds Architecture use nodes, agents, and executors which are distinct from the Jenkins controller itself. Understanding what each of these components are is useful when managing nodes:

Jenkins controller

The Jenkins controller is the Jenkins service itself and is where Jenkins is installed. It is a webserver that also acts as a "brain" for deciding how, when and where to run tasks. Management tasks (configuration, authorization, and authentication) are executed on the controller, which serves HTTP requests. Files written when a Pipeline executes are written to the filesystem on the controller unless they are off-loaded to an artifact repository such as Nexus or Artifactory.

Nodes

Nodes are the "machines" on which build agents run. Jenkins monitors each attached node for disk space, free temp space, free swap, clock time/sync and response time. A node is taken offline if any of these values go outside the configured threshold.

The Jenkins controller itself runs on a special built-in node . It is possible to run agents and executors on this built-in node although this can degrade performance, reduce scalability of the Jenkins instance, and create serious security problems and is strongly discouraged, especially for production environments.

Agents

Agents manage the task execution on behalf of the Jenkins controller by using executors. An agent is actually a small (170KB single jar) Java client process that connects to a Jenkins controller and is assumed to be unreliable. An agent can use any operating system that supports Java. Tools required for builds and tests are installed on the node where the agent runs; they can be installed directly or in a container (Docker or Kubernetes). Each agent is effectively a process with its own PID (Process Identifier) on the host machine.

In practice, nodes and agents are essentially the same but it is good to remember that they are conceptually distinct.

Executors

An executor is a slot for execution of tasks; effectively, it is a thread in the agent. The number of executors on a node defines the number of concurrent tasks that can be executed on that node at one time. In other words, this determines the number of concurrent Pipeline stages that can execute on that node at one time.

The proper number of executors per build node must be determined based on the resources available on the node and the resources required for the workload. When determining how many executors to run on a node, consider CPU and memory requirements as well as the amount of I/O and network activity:

  • One executor per node is the safest configuration.

  • One executor per CPU core may work well if the tasks being run are small.

  • Monitor I/O performance, CPU load, memory usage, and I/O throughput carefully when running multiple executors on a node.

Creating Agents

Jenkins agents are the "workers" that perform operations requested by the Jenkins controller. The Jenkins controller administers the agents and can manage the tooling on the agents. Jenkins agents may be statically allocated or they can be dynamically allocated through systems like Kubernetes, OpenShift, Amazon EC2, Azure, Google Cloud, IBM Cloud, Oracle Cloud, and other cloud providers.

This 30 minute tutorial from Darin Pope creates a Jenkins agent and connects it to a controller.

How to create an agent node in Jenkins

Launch inbound agent via Windows Scheduler

If you are having trouble getting the inbound agent installed as a Windows service (i.e., you followed the instructions on installing the agent as a service here but it didn’t work), an alternative method of starting the service automatically when Windows starts is to use the Windows Scheduler.Â

We take advantage of the Windows Scheduler’s ability to run command at system startup

  1. Configure your node to use the "Launch agents by connecting it to the master" launch method

    • Click Save

  2. Note the command required to launch the agent

    • On the new agent node’s Jenkins page, note the agent command line shown.Â

      • It will be like:

java -jar agent.jar \
-jnlpUrl http://<JenkinsHostName>:8080/computer/<nodeName>/slave-agent.jnlp \
-secret <some_long_hex_string>
  1. Obtain the agent.jar file and copy it to your new Windows agent node

    • In the command line noted in the last step, the "agent.jar" is a hyperlink. Click it to download the agent.jar file.

    • Copy the agent.jar file to a permanent location on your agent machine

  2. Ensure that you have a java version available on your agent machine

    • If not, obtain and install a supported version of Java

  3. Run the command manually from a CMD window on your agent to confirm that it works

    • Open the CMD window

    • Run the command the one like

java -jar agent.jar -jnlpUrl \
http://<JenkinsHostName>:8080/computer/<nodeName>/slave-agent.jnlp -secret <some_long_hex_string>
  • Go back to the node’s web page in Jenkins.  If everything works then page should say "Agent is connected"

  • Stop the command (control-c)

    1. Register a new scheduled job to run the same command

  • Open "Task Scheduler" on your windows machine

    • Start → Run: task Scheduler

  • Create a basic task (Menu: Action → Create Basic Task)

    • First page of the wizard:

      • Name: Jenkins Agent

      • Description (optional)

      • Click Next

    • Next page of the wizard

      • When do you want the task to start: select "When the computer starts"

      • Click Next

    • Next page of the wizard

      • What action do you want the task to perform: select "Start a program"

      • Click Next

    • Next page of the wizard

      • Program/Script: enter "java.exe" (or the full path to your java.exe)

      • Add arguments: enter the rest of the command, like

-jar agent.jar -jnlpUrl http://<JenkinsHostName>:8080/computer/<nodeName>/slave-agent.jnlp \
-secret <some_long_hex_string>
  • eg:

-jar D:\Scripts\jenkins\agent.jar \
-jnlpUrl http://jenkinshost.example.com:8080/computer/buildNode1/slave-agent.jnlp -secret \
d6a84df1fc4f45ddc9c6ab34b08f13391983ffffffffffb3488b7d5ac77fbc7
  • Click Next

    • Next page of the wizard

  • Click the check box "Open the Properties dialog for this task when I click Finish

  • Click Finish

    • Update the task’s properties

      • On the General tab

  • Select the user to run the task as

  • Select "Run whether user is logged on or not"

    • On the settings tab

  • Uncheck "Stop the task if it runs longer than"

  • Check "Run the task as soon as possible after a scheduled start is missed"

  • Check "If the task failed, restart every: 10 minutes", and "Attempt to restart up to: 3 times"

    • Click OK

      1. Start the scheduled task and again check that the agent is connected

        • Go back to the node’s web page in Jenkins.  If everything works then page should say "Agent is connected"

Installing a Jenkins agent on Windows

You can install a Jenkins agent on Windows using the command line. In this video, Darin reviews setting up and installing the Jenkins agent, including how to create any necessary files.

How to install a Jenkins agent on Windows

Creating a macOS agent for Jenkins

This video reviews the process of creating a macOS agent for Jenkins using Java 11.



Was this page helpful?

Please submit your feedback about this page through this quick form.

Alternatively, if you don't wish to complete the quick form, you can simply indicate if you found this page helpful?


See existing feedback here.

Managing Plugins

Managing Plugins

Table of Contents
  • Installing a plugin
    • From the web UI
    • Using the Jenkins CLI
    • Advanced installation
  • Updating a plugin
  • Removing a plugin
    • Uninstalling a plugin
    • Disabling a plugin
  • Pinned plugins

Plugins are the primary means of enhancing the functionality of a Jenkins environment to suit organization- or user-specific needs. There are over a thousand different plugins which can be installed on a Jenkins controller and to integrate various build tools, cloud providers, analysis tools, and much more.

Plugins can be automatically downloaded, with their dependencies, from the Update Center. The Update Center is a service operated by the Jenkins project which provides an inventory of open source plugins which have been developed and maintained by various members of the Jenkins community.

This section covers everything from the basics of managing plugins within the Jenkins web UI, to making changes on the controller’s file system.

Installing a plugin

Jenkins provides two methods for installing plugins on the controller:

  1. Using the "Plugin Manager" in the web UI.

  2. Using the Jenkins CLI install-plugin command.

Each approach will result in the plugin being loaded by Jenkins but may require different levels of access and trade-offs in order to use.

The two approaches require that the Jenkins controller be able to download meta-data from an Update Center, whether the primary Update Center operated by the Jenkins project [1] , or a custom Update Center.

The plugins are packaged as self-contained .hpi files, which have all the necessary code, images, and other resources which the plugin needs to operate successfully.

From the web UI

The simplest and most common way of installing plugins is through the Manage Jenkins > Manage Plugins view, available to administrators of a Jenkins environment.

Under the Available tab, plugins available for download from the configured Update Center can be searched and considered:

Available tab in the Plugin Manager

Most plugins can be installed and used immediately by checking the box adjacent to the plugin and clicking Install without restart .

If the list of available plugins is empty, the controller may be incorrectly configured or has not yet downloaded plugin meta-data from the Update Center. Clicking the Check now button forces Jenkins to attempt to contact its configured Update Center.

Using the Jenkins CLI

Installing Jenkins plugins using Jenkins CLI

Administrators may also use the Jenkins CLI which provides a command to install plugins. Scripts to manage Jenkins environments, or configuration management code, may need to install plugins without direct user interaction in the web UI. The Jenkins CLI allows a command line user or automation tool to download a plugin and its dependencies.

java -jar jenkins-cli.jar -s http://localhost:8080/ install-plugin SOURCE ... [-deploy] [-name VAL] [-restart]

Installs a plugin either from a file, an URL, or from update center.

 SOURCE    : If this points to a local file, that file will be installed. If
             this is an URL, Jenkins downloads the URL and installs that as a
             plugin.Otherwise the name is assumed to be the short name of the
             plugin in the existing update center (like "findbugs"),and the
             plugin will be installed from the update center.
 -deploy   : Deploy plugins right away without postponing them until the reboot.
 -name VAL : If specified, the plugin will be installed as this short name
             (whereas normally the name is inferred from the source name
             automatically).
 -restart  : Restart Jenkins upon successful installation.

Advanced installation

The Update Center only allows the installation of the most recently released version of a plugin. In cases where an older release of the plugin is desired, a Jenkins administrator can download an older .hpi archive and manually install that on the Jenkins controller.

Jenkins stores plugins it has downloaded in the plugins directory with a .jpi suffix, whether the plugins originally had a .jpi or an .hpi suffix.

If an administrator manually copies a plugin archive into the plugins directory, it should be named with a .jpi suffix to match the file names used by plugins installed from the update center.

From the web UI

Assuming a .hpi file has been downloaded, a logged-in Jenkins administrator may upload the file from within the web UI:

  1. Navigate to the Manage Jenkins > Manage Plugins page in the web UI.

  2. Click on the Advanced tab.

  3. Choose the .hpi file from your system or enter a URL to the archive file under the Deploy Plugin section.

  4. Deploy the plugin file.

Advanced tab in the Plugin Manager

Once a plugin file has been uploaded, the Jenkins controller must be manually restarted in order for the changes to take effect.

On the controller

Assuming a .hpi file has been explicitly downloaded by a system administrator, the administrator can manually place the file in a specific location on the file system.

Copy the downloaded .hpi` file into the JENKINS_HOME/plugins directory on the Jenkins controller (for example, on Debian systems JENKINS_HOME is generally /var/lib/jenkins ). If an administrator manually copies a plugin archive into the plugins directory, it should be named with a .jpi suffix to match the file names used by plugins installed from the update center.

The controller must be restarted before the plugin is loaded and made available in the Jenkins environment.

The names of the plugin directories in the Update Site [1] are not always the same as the plugin’s display name. Searching plugins.jenkins.io for the desired plugin will provide the appropriate link to the archive file.

Updating a plugin

Updates are listed in the Updates tab of the Manage Plugins page and can be installed by checking the checkboxes of the desired plugin updates and clicking the Download now and install after restart button.

Updates tab in the Plugin Manager

By default, the Jenkins controller will check for updates from the Update Center once every 24 hours. To manually trigger a check for updates, simply click on the Check now button in the Updates tab.

Removing a plugin

When a plugin is no longer used in a Jenkins environment, it is prudent to remove the plugin from the Jenkins controller. This provides a number of benefits such as reducing memory overhead at boot or runtime, reducing configuration options in the web UI, and removing the potential for future conflicts with new plugin updates.

Uninstalling a plugin

This video reviews the process of uninstalling a plugin from Jenkins.

The simplest way to uninstall a plugin is to navigate to the Installed tab on the Manage Plugins page. From there, Jenkins will automatically determine which plugins are safe to uninstall, those which are not dependencies of other plugins, and present a button for doing so.

Installed tab in the Plugin Manager

A plugin may also be uninstalled by removing the corresponding .jpi file from the JENKINS_HOME/plugins directory on the controller. The plugin will continue to function until the controller has been restarted.

If a plugin file is removed but required by other plugins, the Jenkins controller may fail to boot correctly.

Uninstalling a plugin does not remove the configuration that the plugin may have created. If there are existing jobs/nodes/views/builds/etc configurations that reference data created by the plugin, during boot Jenkins will warn that some configurations could not be fully loaded and ignore the unrecognized data.

Since the configuration(s) will be preserved until they are overwritten, re-installing the plugin will result in those configuration values reappearing.

Removing old data

Jenkins provides a facility for purging configuration left behind by uninstalled plugins. Navigate to Manage Jenkins and then click on Manage Old Data to review and remove old data.

Disabling a plugin

Disabling a plugin is a softer way to retire a plugin. Jenkins will continue to recognize that the plugin is installed, but it will not start the plugin, and no extensions contributed from this plugin will be visible.

A Jenkins administrator may disable a plugin by unchecking the box on the Installed tab of the Manage Plugins page (see below).

Installed tab in the Plugin Manager

A systems administrator may also disable a plugin by creating a file on the Jenkins controller, such as: JENKINS_HOME/plugins/PLUGIN_NAME.jpi.disabled .

The configuration(s) created by the disabled plugin behave as if the plugin were uninstalled, insofar that they result in warnings on boot but are otherwise ignored.

Using the Jenkins CLI

It is also possible to enable or disable plugins via the Jenkins CLI using the enable-plugin or disable-plugin commands.

The enable-plugin command was added to Jenkins in v2.136. The disable-plugin command was added to Jenkins in v2.151.

The enable-plugin command receives a list of plugins to be enabled. Any plugins which a selected plugin depends on will also be enabled by this command.

java -jar jenkins-cli.jar -s http://localhost:8080/ enable-plugin PLUGIN ... [-restart]

Enables one or more installed plugins transitively.

 PLUGIN   : Enables the plugins with the given short names and their
            dependencies.
 -restart : Restart Jenkins after enabling plugins.

The disable-plugin command receives a list of plugins to be disabled. The output will display messages for both successful and failed operations. If you only want to see error messages, the -quiet option can be specified. The -strategy option controls what action will be taken when one of the specified plugins is listed as an optional or mandatory dependency of another enabled plugin.

java -jar jenkins-cli.jar -s http://localhost:8080/ disable-plugin PLUGIN ... [-quiet (-q)]
[-restart (-r)] [-strategy (-s) strategy]

Disable one or more installed plugins.
Disable the plugins with the given short names. You can define how to proceed with the
dependant plugins and if a restart after should be done. You can also set the quiet mode
to avoid extra info in the console.

 PLUGIN                  : Plugins to be disabled.
 -quiet (-q)             : Be quiet, print only the error messages
 -restart (-r)           : Restart Jenkins after disabling plugins.
 -strategy (-s) strategy : How to process the dependant plugins.
                           - none: if a mandatory dependant plugin exists and
                           it is enabled, the plugin cannot be disabled
                           (default value).
                           - mandatory: all mandatory dependant plugins are
                           also disabled, optional dependant plugins remain
                           enabled.
                           - all: all dependant plugins are also disabled, no
                           matter if its dependency is optional or mandatory.

In the same way than enabling and disabling plugins from the UI requires a restart to complete the process, the changes made with the CLI commands will take effect once Jenkins is restarted. The -restart option forces a safe restart of the instance once the command has successfully finished, so the changes will be immediately applied.

Pinned plugins

Pinned plugins feature was removed in Jenkins 2.0. Versions later than Jenkins 2.0 do not bundle plugins, instead providing a wizard to install the most useful plugins.

The notion of pinned plugins applies to plugins that are bundled with Jenkins 1.x, such as the Matrix Authorization plugin .

By default, whenever Jenkins is upgraded, its bundled plugins overwrite the versions of the plugins that are currently installed in JENKINS_HOME .

However, when a bundled plugin has been manually updated, Jenkins will mark that plugin as pinned to the particular version. On the file system, Jenkins creates an empty file called JENKINS_HOME/plugins/PLUGIN_NAME.jpi.pinned to indicate the pinning.

Pinned plugins will never be overwritten by bundled plugins during Jenkins startup. (Newer versions of Jenkins do warn you if a pinned plugin is older than what is currently bundled.)

It is safe to update a bundled plugin to a version offered by the Update Center. This is often necessary to pick up the newest features and fixes. The bundled version is occasionally updated, but not consistently.

The Plugin Manager allows plugins to be explicitly unpinned. The JENKINS_HOME/plugins/PLUGIN_NAME.hpi.pinned file can also be manually created/deleted to control the pinning behavior. If the pinned file is present, Jenkins will use whatever plugin version the user has specified. If the file is absent, Jenkins will restore the plugin to the default version on startup.


1. updates.jenkins.io


Was this page helpful?

Please submit your feedback about this page through this quick form.

Alternatively, if you don't wish to complete the quick form, you can simply indicate if you found this page helpful?


See existing feedback here.

Read article

In-process Script Approval

In-process Script Approval

Table of Contents
  • Getting Started
  • Groovy Sandbox
  • Script Approval
    • Approve assuming permissions check

Jenkins, and a number of plugins, allow users to execute Groovy scripts in Jenkins. These scripting capabilities are provided by:

  • Script Console.

  • Jenkins Pipeline.

  • The Extended Email plugin.

  • The Groovy plugin - when using the "Execute system Groovy script" step.

  • The JobDSL plugin as of version 1.60 and later.

To protect Jenkins from execution of malicious scripts, these plugins execute user-provided scripts in a Groovy Sandbox that limits the internal APIs that are accessible. This protection is provided by the Script Security plugin. As soon as an unsafe method is used in any of the scripts, the administrator can use the "In-process Script Approval" action appears in Manage Jenkins to allow the unsafe method. Unsafe methods should not be enabled without careful consideration of the impact.

Entering the In-process Script Approval configuration

Getting Started

The Script Security plugin is installed automatically by the Post-install Setup Wizard, although initially no additional scripts or operations are approved for use.

Older versions of this plugin may not be safe to use. Please review the security warnings listed on the Script Security plugin page in order to ensure that the Script Security plugin is up to date.

Security for in-process scripting is provided by two different mechanisms: the Groovy Sandbox and Script Approval. The first, the Groovy Sandbox, is enabled by default for Jenkins Pipeline allowing user-supplied Scripted and Declarative Pipeline to execute without prior Administrator intervention. The second, Script Approval, allows Administrators to approve or deny unsandboxed scripts, or allow sandboxed scripts to execute additional methods.

For most instances, the combination of the Groovy Sandbox and the Script Security’s built-in list of approved method signatures, will be sufficient. It is strongly recommended that Administrators only deviate from these defaults if absolutely necessary.

Groovy Sandbox

To reduce manual interventions by Administrators, most scripts will run in a Groovy Sandbox by default, including all Jenkins Pipelines. The sandbox only allows a subset of Groovy’s methods deemed sufficiently safe for "untrusted" access to be executed without prior approval. Scripts using the Groovy Sandbox are all subject to the same restrictions, therefore a Pipeline authored by an Administrator is subject to the restrictions as one authorized by a non-administrative user.

When a script attempts to use features or methods unauthorized by the sandbox, a script is halted immediately, as shown below with Jenkins Pipeline

Sandbox method rejection
Figure 1. Unauthorized method signature rejected at runtime via Blue Ocean

The Pipeline above will not execute until an Administrator approves the method signature via the In-process Script Approval page.

In addition to adding approved method signatures, users may also disable the Groovy Sandbox entirely as shown below. Disabling the Groovy Sandbox requires that the entire script must be reviewed and manually approved by an administrator.

Creating a Scripted Pipeline and unchecking 'Use Groovy Sandbox'
Figure 2. Disabling the Groovy Sandbox for a Pipeline

Script Approval

Manual approval of entire scripts, or method signatures, by an administrator provides Administrators with additional flexibility to support more advanced usages of in-process scripting. When the Groovy Sandbox is disabled, or a method outside of the built-in list is invoked, the Script Security plugin will check the Administrator-managed list of approved scripts and methods.

When a script is approved, it is approved for use in any Jenkins feature or plugin that integrates with script approval. Script approval is not tied to a specific job or to any other specific use of the script. Due to this, care must be taken when approving a script to ensure that any user supplied parameters can not be used to exploit the instance.

For scripts which wish to execute outside of the Groovy Sandbox, the Administrator must approve the entire script in the In-process Script Approval page:

Approving an unsandboxed Scripted Pipeline
Figure 3. Approving an unsandboxed Scripted Pipeline

For scripts which use the Groovy Sandbox, but wish to execute an currently unapproved method signature will also be halted by Jenkins, and require an Administrator to approve the specific method signature before the script is allowed to execute:

Approving a new method signature
Figure 4. Approving a new method signature

Approve assuming permissions check

Script approval provides three options: Approve, Deny, and "Approve assuming permissions check." While the purpose of the first two are self-evident, the third requires some additional understanding of what internal data scripts are able to access and how permissions checks inside of Jenkins function.

Consider a script which accesses the method hudson.model.AbstractItem.getParent() , which by itself is harmless and will return an object containing either the folder or root item which contains the currently executing Pipeline or Job. Following that method invocation, executing hudson.model.ItemGroup.getItems() , which will list items in the folder or root item, requires the Job/Read permission.

This could mean that approving the hudson.model.ItemGroup.getItems() method signature would allow a script to bypass built-in permissions checks.

Instead, it is usually more desirable to click Approve assuming permissions check which will cause the Script Approval engine to allow the method signature assuming the user running the script has the permissions to execute the method, such as the Job/Read permission in this example.



Was this page helpful?

Please submit your feedback about this page through this quick form.

Alternatively, if you don't wish to complete the quick form, you can simply indicate if you found this page helpful?


See existing feedback here.

Read article

Script Console

Script Console

Jenkins features a Groovy script console which allows one to run arbitrary Groovy scripts within the Jenkins controller runtime or in the runtime on agents.

It is very important  to understand all of the following points because it affects the integrity of your Jenkins installation. The Jenkins Script Console:

  • Access is controlled by the Administer  permission.

  • Is a web-based Groovy shell into the Jenkins runtime. Groovy is a very powerful language which offers the ability to do practically anything Java can do including:

    • Create sub-processes and execute arbitrary commands on the Jenkins controller and agents.

    • It can even read files in which the Jenkins controller has access to on the host (like /etc/passwd )

    • Decrypt credentials configured within Jenkins.

  • Offers no administrative controls to stop a User (or Admin) once they are able to execute the Script Console from affecting all parts of the Jenkins infrastructure. Granting a normal Jenkins user Script Console Access is essentially the same as giving them Administrator rights within Jenkins.

  • Can configure any Jenkins setting. It can disable security, reconfigure security, even open a backdoor on the host operating system completely outside of the Jenkins process. Due to the mission critical importance many organizations place on Jenkins in their infrastructure this point is especially important because it would allow an attacker to move laterally within infrastructure with little effort.

  • Is so powerful because it was originally intended as a debugging interface for Jenkins developers but has since grown into an interface used by Jenkins Admins to configure Jenkins and debug Jenkins runtime issues.

Because of the power offered by the Jenkins Script Console, Jenkins and its agents should never be run as the root  user (on Linux) or system administrator on any other flavor of OS. Videos linked in this page demonstrate and discuss security warnings.

Be sure to secure your Jenkins instance

Table of Contents
  • Multiple contexts
    • Running Script Console on the controller
    • Running Script Console on agents
    • Run scripts from controller Script Console on agents
    • Reading and writing files
  • Remote access
  • Shortcut key on script console to submit
  • Video Tutorials and additional learning materials
  • Example Groovy scripts
    • Out of date scripts
  • Plugins enabling Groovy usage

Multiple contexts

The Jenkins Script Console can run either on the controller or any configured agents.

Running Script Console on the controller

This feature can be accessed from "Manage Jenkins" > "Script Console" .  Or by visiting the sub-URL /script  on your Jenkins instance.

Running Script Console on agents

Visit "Manage Jenkins" > "Manage Nodes" .  Select any node to view the status page.  In the menu on the left, a menu item is available to open a "Script Console" on that specific agent.

Run scripts from controller Script Console on agents

It’s also possible to run scripts from the controller Script Console on individual agents.  The following script is an example running a script on agents from the controller Script Console.

Script executes code on agent from Master Script Console

import hudson.util.RemotingDiagnostics
import jenkins.model.Jenkins

String agentName = 'your agent name'
//groovy script you want executed on an agent
groovy_script = '''
println System.getenv("PATH")
println "uname -a".execute().text
'''.trim()

String result
Jenkins.instance.slaves.find { agent ->
    agent.name == agentName
}.with { agent ->
    result = RemotingDiagnostics.executeGroovy(groovy_script, agent.channel)
}
println result

Reading and writing files

Files can be read and written directly on the controller or agents via the controller Script Console.

Write a file to the Jenkins controller

new File('/tmp/file.txt').withWriter('UTF-8') { writer ->
    try {
        writer << 'hello world\n'
    } finally {
        writer.close()
    }
}

Reading a file from the Jenkins controller

new File('/tmp/file.txt').text

Write file to agent through agent channel

import hudson.FilePath
import hudson.remoting.Channel
import jenkins.model.Jenkins

String agentName = 'some-agent'
String filePath = '/tmp/file.txt'

Channel agentChannel = Jenkins.instance.slaves.find { agent ->
    agent.name == agentName
}.channel

new FilePath(agentChannel, filePath).write().with { os ->
    try {
        os << 'hello world\n'
    } finally {
        os.close()
    }
}

Read file from agent through agent channel

import hudson.FilePath
import hudson.remoting.Channel
import jenkins.model.Jenkins

import java.io.BufferedReader
import java.io.InputStreamReader
import java.nio.charset.StandardCharsets
import java.util.stream.Collectors

String agentName = 'some-agent'
String filePath = '/tmp/file.txt'

Channel agentChannel = Jenkins.instance.slaves.find { agent ->
    agent.name == agentName
}.channel

String fileContents = ''
new FilePath(agentChannel, filePath).read().with { is ->
    try {
        fileContents = new BufferedReader(
            new InputStreamReader(is, StandardCharsets.UTF_8))
                .lines()
                .collect(Collectors.joining("\n"))
    } finally {
        is.close()
    }
}

// print contents of the file from the agent
println '==='
println(fileContents)
println '==='

Remote access

A Jenkins Admin can execute groovy scripts remotely by sending an HTTP POST request to /script/ url or /scriptText/ .

curl example via bash

curl -d "script=<your_script_here>" https://jenkins/script
# or to get output as a plain text result (no HTML)
curl -d "script=<your_script_here>" https://jenkins/scriptText

Also, Jenkins CLI offers the possibility to execute groovy scripts remotely using groovy command or execute groovy interactively via groovysh . However, once again curl can be used to execute groovy scripts by making use of bash command substitution. In the following example somescript.groovy is a groovy script in the current working directory.

Curl submitting groovy file via bash

curl --data-urlencode "script=$(< ./somescript.groovy)" https://jenkins/scriptText

If security is configured in Jenkins, then curl can be provided options to authenticate using the curl --user  option.

Curl submitting groovy file providing username and api token via bash

curl --user 'username:api-token' --data-urlencode \
  "script=$(< ./somescript.groovy)" https://jenkins/scriptText

Here is the equivalent command using python, not curl.

Python submitting groovy file providing username and api token

with open('somescript.groovy', 'r') as fd:
    data = fd.read()
r = requests.post('https://jenkins/scriptText', auth=('username', 'api-token'), data={'script': data})

Shortcut key on script console to submit

You can submit a script without mouse. Jenkins has a shortcut key which enables to submit with keyboard.

  • Windows / Linux: Ctrl + Enter

  • Mac: Command + Enter

Video Tutorials and additional learning materials

Here are some recorded videos on the Jenkins Script Console:

To expand your ability to write scripts in the script console, the following references are recommended:

  • Learn Groovy - Learning Groovy is useful for more than writing scripts for the Script Console.  Groovy is also relevant for other features of Jenkins like Pipelines and shared pipeline libraries , the Groovy Plugin , the Job DSL plugin, and many other plugins which utilize Groovy (see section [Plugins-enabling-Groovy-usage]).

  • Write Groovy scripts for Jenkins with Code completion  - The gist of this is to create a Maven project within your IDE and to depend on org.jenkins-ci.main:jenkins-core (and any other plugins that you expect present). You can then write a Groovy script with code completion of Jenkins API objects and methods.

Example Groovy scripts

Out of date scripts

Due to the nature of Groovy scripts accessing Jenkins source code directly, Script Console scripts are easily out of date from the Jenkins source code. It is possible to run a script and get exceptions because public methods and interfaces in Jenkins core or Jenkins plugins have changed. Keep this in mind when trying out examples. Jenkins is easily started from a local development machine via the following command:

Starting a local copy of Jenkins

export JENKINS_HOME="./my_jenkins_home"
java -jar jenkins.war

Use CTRL+C to stop Jenkins. It is not recommended to try Script Console examples in a production Jenkins instance.

The following repositories offer solid examples of Groovy scripts for Jenkins.

Browse all Scriptler Plugin Groovy Scripts and please share your scripts with the Scriptler Plugin.

Plugins enabling Groovy usage

  • Config File Provider Plugin Adds the ability to provide configuration files (i.e., settings.xml for maven, XML, groovy, custom files, etc.) loaded through the Jenkins UI which will be copied to the job’s workspace.

  • Global Post Script Plugin — Execute a global configured groovy script after each build of each job managed by the Jenkins. This is typical for cases when you need to do something based on a shared set of parameters, such as triggering downstream jobs managed by the same Jenkins or remote ones based on the parameters been passed to the parameterized jobs.

  • Groovy plugin

  • Groovy Postbuild Plugin — This plugin executes a groovy script in the Jenkins JVM. Typically, the script checks some conditions and changes accordingly the build result, puts badges next to the build in the build history and/or displays information on the build summary page.

  • Groovy Remote Control Plugin — This plugin provides Groovy Remote Control's receiver, and allows to control external application from Jenkins.

  • Matrix Groovy Execution Strategy Plugin — A plugin to decide the execution order and valid combinations of matrix projects.

  • Pipeline Classpath Step Plugin Pipeline DSL step to add path to the groovy classpath

  • Scriptler Plugin — Scriptler allows you to store/edit groovy scripts and execute it on any of the nodes…​ no need to copy/paste groovy code anymore.



Was this page helpful?

Please submit your feedback about this page through this quick form.

Alternatively, if you don't wish to complete the quick form, you can simply indicate if you found this page helpful?


See existing feedback here.

Read article

Spawning Processes From Build

Spawning Processes From Build

Table of Contents
  • Why?
  • Workarounds

It is possible to spawn a process from a build and have that process live longer than the build itself. For example, perhaps the build launches a new application server with the result of the build. In older releases, the build often did not terminate. Instead, the specific step (such as the shell script, Ant, or Maven) terminates but the build itself does not terminate.

Jenkins detects this situation and, instead of blocking indefinitely, prints out a warning and terminates the build.

Why?

This happens because of how file descriptors are used between processes in a build. Jenkins and the child process are connected by three pipes ( stdin , stdout , and stderr .) This allows Jenkins to capture the output from the child process. The child process may write a lot of data to the pipe and quit immediately after that, so Jenkins waits for end-of-file (EOF) to be sure that it has drained the pipes before it terminates the build.

Whenever a process terminates, the operating system closes all the file descriptors it owned. So, even if the process did not close stdout and stderr , Jenkins gets end of file (EOF).

The complication happens when those file descriptors are inherited by other processes. Let’s say the child process forks another process to the background. The background process (which is actually a daemon) inherits all the file descriptors of the parent, including the writing side of the stdout ad stderr pipes that connect the child process and Jenkins. If the daemon forgets to close them, Jenkins does not get EOF for pipes even when the child process exits, because the daemon still has those descriptors open. This is how this problem happens.

A daemon should close all file descriptors to avoid such issues but some daemons do not follow the rule. You can mitigate this problem with various workarounds.

Workarounds

On Unix, you can use a wrapper like this to make the daemon behave. For example:

daemonize -E BUILD_ID=dontKillMe /path/to/your/command

In a Jenkins Pipeline, use JENKINS_NODE_COOKIE instead of BUILD_ID .

Note that this will set the BUILD_ID environment variable for the process being spawned to something other than the current BUILD_ID. Or you can start jenkins with -Dhudson.util.ProcessTree.disable=true - see long running agent process for details.

On Windows, use the 'at' command to launch a process in the background. For example:

<scriptdef name="get-next-minute" language="beanshell">
  <attribute name="property" />

  date = new java.text.SimpleDateFormat("HH:mm")
    .format(new Date(System.currentTimeMillis() + 60000));
  project.setProperty(attributes.get("property"), date);
</scriptdef>

<get-next-minute property="next-minute" />
<exec executable="at">
  <arg value="${next-minute}" />
  <arg value="/interactive" />
  <arg value="${jboss.home}\bin\run.bat" />
</exec>

Another similar workaround on Windows is to use a wrapper script and launch your program through it:

// antRunAsync.js - Wrapper script to run an executable detached in the
// background from Ant's <exec> task.  This works by running the executable
// using the Windows Scripting Host WshShell.Run method which doesn't copy
// the standard filehandles stdin, stdout and stderr. Ant finds them closed
// and doesn't wait for the program to exit.
//
// requirements:
//   Windows Scripting Host 1.0 or better.  This is included with Windows
//   98/Me/2000/XP.  Users of Windows 95 or Windows NT 4.0 need to download
//   and install WSH support from
//   http://msdn.microsoft.com/scripting/.
//
// usage:
// <exec executable="cscript.exe">
//   <env key="ANTRUN_TITLE" value="Title for Window" />  <!-- optional -->
//   <env key="ANTRUN_OUTPUT" value="output.log" />  <!-- optional -->
//   <arg value="//NoLogo" />
//   <arg value="antRunAsync.js" />  <!-- this script -->
//   <arg value="real executable" />
// </exec>


var WshShell = WScript.CreateObject("WScript.Shell");
var exeStr = "%comspec% /c";
var arg = "";
var windowStyle = 1;
var WshProcessEnv = WshShell.Environment("PROCESS");
var windowTitle = WshProcessEnv("ANTRUN_TITLE");
var outputFile = WshProcessEnv("ANTRUN_OUTPUT");
var OS = WshProcessEnv("OS");
var isWindowsNT = (OS == "Windows_NT");

// On Windows NT/2000/XP, specify a title for the window.  If the environment
// variable ANTRUN_TITLE is specified, that will be used instead of a default.
if (isWindowsNT) {
  if (windowTitle == "")
     windowTitle = "Ant - " + WScript.Arguments(i);
  exeStr += "title " + windowTitle + " &&";
}

// Loop through arguments quoting ones with spaces
for (var i = 0; i < WScript.Arguments.count(); i++) {
  arg = WScript.Arguments(i);
  if (arg.indexOf(' ') > 0)
    exeStr += " \"" + arg + "\"";
  else
    exeStr += " " + arg;
}

// If the environment variable ANTRUN_OUTPUT was specified, redirect
// output to that file.
if (outputFile != "") {
  windowStyle = 7;  // new window is minimized
  exeStr += " > \"" + outputFile + "\"";
  if (isWindowsNT)
    exeStr += " 2>&1";
}

// WScript.Echo(exeStr);
// WshShell.Run(exeStr);
WshShell.Run(exeStr, windowStyle, false);
<exec executable="cscript.exe">
   <env key="ANTRUN_TITLE" value="Title for Window" />  <!-- optional -->
   <env key="ANTRUN_OUTPUT" value="output.log" />  <!-- optional -->
   <arg value="//NoLogo" />
   <arg value="antRunAsync.js" />  <!-- this script -->
   <arg value="real executable" />
</exec>

Another workaround for Windows is to schedule a permanent task and force running it from the Ant script. For example, run the command:

C:\>SCHTASKS /Create /RU SYSTEM /SC ONSTART /TN Tomcat /TR
"C:\Program Files\Apache Software Foundation\Tomcat 6.0\bin\startup.bat"

Note, that ONSTART can be replaced with ONCE if you do not want to keep Tomcat running. Add the following code to your Ant script:

<exec executable="SCHTASKS">
    <arg value="/Run"/>
    <arg value="/TN"/>
    <arg value="Tomcat"/>
</exec>


Was this page helpful?

Please submit your feedback about this page through this quick form.

Alternatively, if you don't wish to complete the quick form, you can simply indicate if you found this page helpful?


See existing feedback here.

Read article

Configuring the System

Configuring the System

This section is a work in progress. Want to help? Check out the jenkinsci-docs mailing list. For other ways to contribute to the Jenkins project, see this page about participating and contributing.



Was this page helpful?

Please submit your feedback about this page through this quick form.

Alternatively, if you don't wish to complete the quick form, you can simply indicate if you found this page helpful?


See existing feedback here.

Read article