Builds in a Distributed Builds Architecture use nodes, agents, and executors which are distinct from the Jenkins controller itself. Understanding what each of these components are is useful when managing nodes:

The Jenkins controller itself runs on a special built-in node . It is possible to run agents and executors on this built-in node although this can degrade performance, reduce scalability of the Jenkins instance, and create serious security problems and is strongly discouraged, especially for production environments.

In practice, nodes and agents are essentially the same but it is good to remember that they are conceptually distinct.

The proper number of executors per build node must be determined based on the resources available on the node and the resources required for the workload. When determining how many executors to run on a node, consider CPU and memory requirements as well as the amount of I/O and network activity:

Jenkins agents are the "workers" that perform operations requested by the Jenkins controller. The Jenkins controller administers the agents and can manage the tooling on the agents. Jenkins agents may be statically allocated or they can be dynamically allocated through systems like Kubernetes, OpenShift, Amazon EC2, Azure, Google Cloud, IBM Cloud, Oracle Cloud, and other cloud providers.

This 30 minute tutorial from Darin Pope creates a Jenkins agent and connects it to a controller.

If you are having trouble getting the inbound agent installed as a Windows service (i.e., you followed the instructions on installing the agent as a service here but it didn’t work), an alternative method of starting the service automatically when Windows starts is to use the Windows Scheduler.Â

We take advantage of the Windows Scheduler’s ability to run command at system startup

You can install a Jenkins agent on Windows using the command line. In this video, Darin reviews setting up and installing the Jenkins agent, including how to create any necessary files.

This video reviews the process of creating a macOS agent for Jenkins using Java 11.

Jenkins provides two methods for installing plugins on the controller:

Each approach will result in the plugin being loaded by Jenkins but may require different levels of access and trade-offs in order to use.

The two approaches require that the Jenkins controller be able to download meta-data from an Update Center, whether the primary Update Center operated by the Jenkins project ^[1] , or a custom Update Center.

The plugins are packaged as self-contained .hpi files, which have all the necessary code, images, and other resources which the plugin needs to operate successfully.

Updates are listed in the Updates tab of the Manage Plugins page and can be installed by checking the checkboxes of the desired plugin updates and clicking the Download now and install after restart button.

By default, the Jenkins controller will check for updates from the Update Center once every 24 hours. To manually trigger a check for updates, simply click on the Check now button in the Updates tab.

When a plugin is no longer used in a Jenkins environment, it is prudent to remove the plugin from the Jenkins controller. This provides a number of benefits such as reducing memory overhead at boot or runtime, reducing configuration options in the web UI, and removing the potential for future conflicts with new plugin updates.

The notion of pinned plugins applies to plugins that are bundled with Jenkins 1.x, such as the Matrix Authorization plugin .

By default, whenever Jenkins is upgraded, its bundled plugins overwrite the versions of the plugins that are currently installed in JENKINS_HOME .

However, when a bundled plugin has been manually updated, Jenkins will mark that plugin as pinned to the particular version. On the file system, Jenkins creates an empty file called JENKINS_HOME/plugins/PLUGIN_NAME.jpi.pinned to indicate the pinning.

Pinned plugins will never be overwritten by bundled plugins during Jenkins startup. (Newer versions of Jenkins do warn you if a pinned plugin is older than what is currently bundled.)

It is safe to update a bundled plugin to a version offered by the Update Center. This is often necessary to pick up the newest features and fixes. The bundled version is occasionally updated, but not consistently.

The Plugin Manager allows plugins to be explicitly unpinned. The JENKINS_HOME/plugins/PLUGIN_NAME.hpi.pinned file can also be manually created/deleted to control the pinning behavior. If the pinned file is present, Jenkins will use whatever plugin version the user has specified. If the file is absent, Jenkins will restore the plugin to the default version on startup.

The Script Security plugin is installed automatically by the Post-install Setup Wizard, although initially no additional scripts or operations are approved for use.

Security for in-process scripting is provided by two different mechanisms: the Groovy Sandbox and Script Approval. The first, the Groovy Sandbox, is enabled by default for Jenkins Pipeline allowing user-supplied Scripted and Declarative Pipeline to execute without prior Administrator intervention. The second, Script Approval, allows Administrators to approve or deny unsandboxed scripts, or allow sandboxed scripts to execute additional methods.

For most instances, the combination of the Groovy Sandbox and the Script Security’s built-in list of approved method signatures, will be sufficient. It is strongly recommended that Administrators only deviate from these defaults if absolutely necessary.

To reduce manual interventions by Administrators, most scripts will run in a Groovy Sandbox by default, including all Jenkins Pipelines. The sandbox only allows a subset of Groovy’s methods deemed sufficiently safe for "untrusted" access to be executed without prior approval. Scripts using the Groovy Sandbox are all subject to the same restrictions, therefore a Pipeline authored by an Administrator is subject to the restrictions as one authorized by a non-administrative user.

When a script attempts to use features or methods unauthorized by the sandbox, a script is halted immediately, as shown below with Jenkins Pipeline

The Pipeline above will not execute until an Administrator approves the method signature via the In-process Script Approval page.

In addition to adding approved method signatures, users may also disable the Groovy Sandbox entirely as shown below. Disabling the Groovy Sandbox requires that the entire script must be reviewed and manually approved by an administrator.

Manual approval of entire scripts, or method signatures, by an administrator provides Administrators with additional flexibility to support more advanced usages of in-process scripting. When the Groovy Sandbox is disabled, or a method outside of the built-in list is invoked, the Script Security plugin will check the Administrator-managed list of approved scripts and methods.

For scripts which wish to execute outside of the Groovy Sandbox, the Administrator must approve the entire script in the In-process Script Approval page:

For scripts which use the Groovy Sandbox, but wish to execute an currently unapproved method signature will also be halted by Jenkins, and require an Administrator to approve the specific method signature before the script is allowed to execute:

The Jenkins Script Console can run either on the controller or any configured agents.

A Jenkins Admin can execute groovy scripts remotely by sending an HTTP POST request to /script/ url or /scriptText/ .

curl example via bash

Also, Jenkins CLI offers the possibility to execute groovy scripts remotely using groovy command or execute groovy interactively via groovysh . However, once again curl can be used to execute groovy scripts by making use of bash command substitution. In the following example somescript.groovy is a groovy script in the current working directory.

Curl submitting groovy file via bash

If security is configured in Jenkins, then curl can be provided options to authenticate using theÂ curl --user  option.

Curl submitting groovy file providing username and api token via bash

Here is the equivalent command using python, not curl.

Python submitting groovy file providing username and api token

You can submit a script without mouse. Jenkins has a shortcut key which enables to submit with keyboard.

Here are some recorded videos on the Jenkins Script Console:

To expand your ability to write scripts in the script console, the following references are recommended:

This happens because of how file descriptors are used between processes in a build. Jenkins and the child process are connected by three pipes ( stdin , stdout , and stderr .) This allows Jenkins to capture the output from the child process. The child process may write a lot of data to the pipe and quit immediately after that, so Jenkins waits for end-of-file (EOF) to be sure that it has drained the pipes before it terminates the build.

Whenever a process terminates, the operating system closes all the file descriptors it owned. So, even if the process did not close stdout and stderr , Jenkins gets end of file (EOF).

The complication happens when those file descriptors are inherited by other processes. Let’s say the child process forks another process to the background. The background process (which is actually a daemon) inherits all the file descriptors of the parent, including the writing side of the stdout ad stderr pipes that connect the child process and Jenkins. If the daemon forgets to close them, Jenkins does not get EOF for pipes even when the child process exits, because the daemon still has those descriptors open. This is how this problem happens.

A daemon should close all file descriptors to avoid such issues but some daemons do not follow the rule. You can mitigate this problem with various workarounds.

On Unix, you can use a wrapper like this to make the daemon behave. For example:

In a Jenkins Pipeline, use JENKINS_NODE_COOKIE instead of BUILD_ID .

Note that this will set the BUILD_ID environment variable for the process being spawned to something other than the current BUILD_ID. Or you can start jenkins with -Dhudson.util.ProcessTree.disable=true - see long running agent process for details.

On Windows, use the 'at' command to launch a process in the background. For example:

Another similar workaround on Windows is to use a wrapper script and launch your program through it:

Another workaround for Windows is to schedule a permanent task and force running it from the Ant script. For example, run the command:

Note, that ONSTART can be replaced with ONCE if you do not want to keep Tomcat running. Add the following code to your Ant script: