This is one stop global knowledge base where you can learn about all the products, solutions and support features.
As shown in the File definition page,
bake
supports variable blocks which are assigned to matching environment variables
or default values:
# docker-bake.hcl
variable "TAG" {
default = "latest"
}
group "default" {
targets = ["webapp"]
}
target "webapp" {
tags = ["docker.io/username/webapp:${TAG}"]
}
alternatively, in json format:
{
"variable": {
"TAG": {
"default": "latest"
}
},
"group": {
"default": {
"targets": ["webapp"]
}
},
"target": {
"webapp": {
"tags": ["docker.io/username/webapp:${TAG}"]
}
}
}
$ docker buildx bake --print webapp
{
"group": {
"default": {
"targets": [
"webapp"
]
}
},
"target": {
"webapp": {
"context": ".",
"dockerfile": "Dockerfile",
"tags": [
"docker.io/username/webapp:latest"
]
}
}
}
$ TAG=$(git rev-parse --short HEAD) docker buildx bake --print webapp
{
"group": {
"default": {
"targets": [
"webapp"
]
}
},
"target": {
"webapp": {
"context": ".",
"dockerfile": "Dockerfile",
"tags": [
"docker.io/username/webapp:985e9e9"
]
}
}
}
add
function
You can use
go-cty
stdlib functions.
Here we are using the
add
function.
# docker-bake.hcl
variable "TAG" {
default = "latest"
}
group "default" {
targets = ["webapp"]
}
target "webapp" {
args = {
buildno = "${add(123, 1)}"
}
}
$ docker buildx bake --print webapp
{
"group": {
"default": {
"targets": [
"webapp"
]
}
},
"target": {
"webapp": {
"context": ".",
"dockerfile": "Dockerfile",
"args": {
"buildno": "124"
}
}
}
}
increment
function
It also supports user defined functions.
The following example defines a simple an
increment
function.
# docker-bake.hcl
function "increment" {
params = [number]
result = number + 1
}
group "default" {
targets = ["webapp"]
}
target "webapp" {
args = {
buildno = "${increment(123)}"
}
}
$ docker buildx bake --print webapp
{
"group": {
"default": {
"targets": [
"webapp"
]
}
},
"target": {
"webapp": {
"context": ".",
"dockerfile": "Dockerfile",
"args": {
"buildno": "124"
}
}
}
}
notequal
Here we are using the conditional
notequal
function which is just for
symmetry with the
equal
one.
# docker-bake.hcl
variable "TAG" {default="" }
group "default" {
targets = [
"webapp",
]
}
target "webapp" {
context="."
dockerfile="Dockerfile"
tags = [
"my-image:latest",
notequal("",TAG) ? "my-image:${TAG}": "",
]
}
$ docker buildx bake --print webapp
{
"group": {
"default": {
"targets": [
"webapp"
]
}
},
"target": {
"webapp": {
"context": ".",
"dockerfile": "Dockerfile",
"tags": [
"my-image:latest"
]
}
}
}
You can refer variables to other variables like the target blocks can. Stdlib functions can also be called but user functions canât at the moment.
# docker-bake.hcl
variable "REPO" {
default = "user/repo"
}
function "tag" {
params = [tag]
result = ["${REPO}:${tag}"]
}
target "webapp" {
tags = tag("v1")
}
$ docker buildx bake --print webapp
{
"group": {
"default": {
"targets": [
"webapp"
]
}
},
"target": {
"webapp": {
"context": ".",
"dockerfile": "Dockerfile",
"tags": [
"user/repo:v1"
]
}
}
}
Non-string variables are also accepted. The value passed with env is parsed into suitable type first.
# docker-bake.hcl
variable "FOO" {
default = 3
}
variable "IS_FOO" {
default = true
}
target "app" {
args = {
v1 = FOO > 5 ? "higher" : "lower"
v2 = IS_FOO ? "yes" : "no"
}
}
$ docker buildx bake --print app
{
"group": {
"default": {
"targets": [
"app"
]
}
},
"target": {
"app": {
"context": ".",
"dockerfile": "Dockerfile",
"args": {
"v1": "lower",
"v2": "yes"
}
}
}
}
This command is experimental.
The design of bake is in early stages, and we are looking for feedback from users.
Buildx also aims to provide support for high-level build concepts that go beyond invoking a single build command. We want to support building all the images in your application together and let the users define project specific reusable build flows that can then be easily invoked by anyone.
BuildKit
efficiently handles multiple concurrent build requests and de-duplicating work.
The build commands can be combined with general-purpose command runners
(for example,
make
). However, these tools generally invoke builds in sequence
and therefore cannot leverage the full potential of BuildKit parallelization,
or combine BuildKitâs output for the user. For this use case, we have added a
command called
docker buildx bake
.
The
bake
command supports building images from HCL, JSON and Compose files.
This is similar to
docker compose build
,
but allowing all the services to be built concurrently as part of a single
request. If multiple files are specified they are all read and configurations are
combined.
We recommend using HCL files as its experience is more aligned with buildx UX and also allows better code reuse, different target groups and extended features.
Most Dockerfiles start from a parent image. If you need to completely control the contents of your image, you might need to create a base image instead. Hereâs the difference:
A parent image is the image that your
image is based on. It refers to the contents of the
FROM
directive in the
Dockerfile. Each subsequent declaration in the Dockerfile modifies this parent
image. Most Dockerfiles start from a parent image, rather than a base image.
However, the terms are sometimes used interchangeably.
A base image has
FROM scratch
in its Dockerfile.
This topic shows you several ways to create a base image. The specific process will depend heavily on the Linux distribution you want to package. We have some examples below, and you are encouraged to submit pull requests to contribute new ones.
In general, start with a working machine that is running the distribution youâd like to package as a parent image, though that is not required for some tools like Debianâs Debootstrap, which you can also use to build Ubuntu images.
It can be as simple as this to create an Ubuntu parent image:
$ sudo debootstrap focal focal > /dev/null
$ sudo tar -C focal -c . | docker import - focal
sha256:81ec9a55a92a5618161f68ae691d092bf14d700129093158297b3d01593f4ee3
$ docker run focal cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04 LTS"
There are more example scripts for creating parent images in the Docker GitHub repository.
You can use Dockerâs reserved, minimal image,
scratch
, as a starting point for
building containers. Using the
scratch
âimageâ signals to the build process
that you want the next command in the
Dockerfile
to be the first filesystem
layer in your image.
While
scratch
appears in Dockerâs repository on the hub, you canât pull it,
run it, or tag any image with the name
scratch
. Instead, you can refer to it
in your
Dockerfile
. For example, to create a minimal container using
scratch
:
# syntax=docker/dockerfile:1
FROM scratch
ADD hello /
CMD ["/hello"]
Assuming you built the âhelloâ executable example by using the source code at
https://github.com/docker-library/hello-world,
and you compiled it with the
-static
flag, you can build this Docker
image using this
docker build
command:
$ docker build --tag hello .
Donât forget the
.
character at the end, which sets the build context
to the current directory.
Note : Because Docker Desktop for Mac and Docker Desktop for Windows use a Linux VM, you need a Linux binary, rather than a Mac or Windows binary. You can use a Docker container to build it:
$ docker run --rm -it -v $PWD:/build ubuntu:20.04 container# apt-get update && apt-get install build-essential container# cd /build container# gcc -o hello -static hello.c
To run your new image, use the
docker run
command:
$ docker run --rm hello
This example creates the hello-world image used in the tutorials. If you want to test it out, you can clone the image repo.
There are lots of resources available to help you write your
Dockerfile
.
Dockerfile
in the reference section.
Dockerfile
, weâve also
written a Dockerfile best practices guide.
The
docker build
or
docker buildx build
commands build Docker images from a Dockerfile
and a âcontextâ.
A buildâs context is the set of files located at the
PATH
or
URL
specified
as the positional argument to the build command:
$ docker build [OPTIONS] PATH | URL | -
^^^^^^^^^^^^^^
The build process can refer to any of the files in the context. For example,
your build can use a
COPY
instruction
to reference a file in the context or a
RUN --mount=type=bind
instruction
for better performance with BuildKit. The build context
is processed recursively. So, a
PATH
includes any subdirectories and the
URL
includes the repository and its submodules.
PATH
context
This example shows a build command that uses the current directory (
.
) as a
build context:
$ docker build .
...
#16 [internal] load build context
#16 sha256:23ca2f94460dcbaf5b3c3edbaaa933281a4e0ea3d92fe295193e4df44dc68f85
#16 transferring context: 13.16MB 2.2s done
...
With the following Dockerfile:
# syntax=docker/dockerfile:1
FROM busybox
WORKDIR /src
COPY foo .
And this directory structure:
.
âââ Dockerfile
âââ bar
âââ foo
âââ node_modules
The legacy builder sends the entire directory to the daemon, including
bar
and
node_modules
directories, even though the
Dockerfile
does not use
them. When using BuildKit, the client only sends the
files required by the
COPY
instructions, in this case
foo
.
In some cases you may want to send the entire context:
# syntax=docker/dockerfile:1
FROM busybox
WORKDIR /src
COPY . .
You can use a
.dockerignore
file to exclude some files or directories from being sent:
# .dockerignore
node_modules
bar
Warning
Avoid using your root directory,
/
, as thePATH
for your build context, as it causes the build to transfer the entire contents of your hard drive to the daemon.
URL
context
The
URL
parameter can refer to three kinds of resources:
When the
URL
parameter points to the location of a Git repository, the
repository acts as the build context. The builder recursively pulls the
repository and its submodules. A shallow clone is performed and therefore pulls
down just the latest commits, not the entire history. A repository is first
pulled into a temporary directory on your host. After that succeeds, the
directory is sent to the daemon as the context. Local copy gives you the ability
to access private repositories using local user credentials, VPNâs, and so forth.
Note
If the
URL
parameter contains a fragment the system will recursively clone the repository and its submodules using agit clone --recursive
command.
Git URLs accept a context configuration parameter in the form of a URL fragment,
separated by a colon (
:
). The first part represents the reference that Git
will check out, and can be either a branch, a tag, or a remote reference. The
second part represents a subdirectory inside the repository that will be used
as a build context.
For example, run this command to use a directory called
docker
in the branch
container
:
$ docker build https://github.com/user/myrepo.git#container:docker
The following table represents all the valid suffixes with their build contexts:
Build Syntax Suffix | Commit Used | Build Context Used |
---|---|---|
myrepo.git
|
refs/heads/master
|
/
|
myrepo.git#mytag
|
refs/tags/mytag
|
/
|
myrepo.git#mybranch
|
refs/heads/mybranch
|
/
|
myrepo.git#pull/42/head
|
refs/pull/42/head
|
/
|
myrepo.git#:myfolder
|
refs/heads/master
|
/myfolder
|
myrepo.git#master:myfolder
|
refs/heads/master
|
/myfolder
|
myrepo.git#mytag:myfolder
|
refs/tags/mytag
|
/myfolder
|
myrepo.git#mybranch:myfolder
|
refs/heads/mybranch
|
/myfolder
|
By default
.git
directory is not kept on Git checkouts. You can set the
BuildKit built-in arg
BUILDKIT_CONTEXT_KEEP_GIT_DIR=1
to keep it. It can be useful to keep it around if you want to retrieve Git
information during your build:
# syntax=docker/dockerfile:1
FROM alpine
WORKDIR /src
RUN --mount=target=. \
make REVISION=$(git rev-parse HEAD) build
$ docker build --build-arg BUILDKIT_CONTEXT_KEEP_GIT_DIR=1 https://github.com/user/myrepo.git#main
If you pass a
URL
to a remote tarball, the
URL
itself is sent to the daemon:
$ docker build http://server/context.tar.gz
#1 [internal] load remote build context
#1 DONE 0.2s
#2 copy /context /
#2 DONE 0.1s
...
The download operation will be performed on the host the daemon is running on,
which is not necessarily the same host from which the build command is being
issued. The daemon will fetch
context.tar.gz
and use it as the build context.
Tarball contexts must be tar archives conforming to the standard
tar
UNIX
format and can be compressed with any one of the
xz
,
bzip2
,
gzip
or
identity
(no compression) formats.
Instead of specifying a context, you can pass a single
Dockerfile
in the
URL
or pipe the file in via
STDIN
. To pipe a
Dockerfile
from
STDIN
:
$ docker build - < Dockerfile
With Powershell on Windows, you can run:
Get-Content Dockerfile | docker build -
If you use
STDIN
or specify a
URL
pointing to a plain text file, the system
places the contents into a file called
Dockerfile
, and any
-f
,
--file
option is ignored. In this scenario, there is no context.
The following example builds an image using a
Dockerfile
that is passed
through stdin. No files are sent as build context to the daemon.
docker build -t myimage:latest -<<EOF
FROM busybox
RUN echo "hello world"
EOF
Omitting the build context can be useful in situations where your
Dockerfile
does not require files to be copied into the image, and improves the build-speed,
as no files are sent to the daemon.
Docker images can support multiple platforms, which means that a single image may contain variants for different architectures, and sometimes for different operating systems, such as Windows.
When running an image with multi-platform support,
docker
automatically
selects the image that matches your OS and architecture.
Most of the Docker Official Images on Docker Hub provide a variety of architectures.
For example, the
busybox
image supports
amd64
,
arm32v5
,
arm32v6
,
arm32v7
,
arm64v8
,
i386
,
ppc64le
, and
s390x
. When running this image
on an
x86_64
/
amd64
machine, the
amd64
variant is pulled and run.
Docker is now making it easier than ever to develop containers on, and for Arm servers and devices. Using the standard Docker tooling and processes, you can start to build, push, pull, and run images seamlessly on different compute architectures. In most cases, you donât have to make any changes to Dockerfiles or source code to start building for Arm.
BuildKit with Buildx is designed to work well for building for multiple platforms and not only for the architecture and operating system that the user invoking the build happens to run.
When you invoke a build, you can set the
--platform
flag to specify the target
platform for the build output, (for example,
linux/amd64
,
linux/arm64
, or
darwin/amd64
).
When the current builder instance is backed by the
docker-container
driver,
you can specify multiple platforms together. In this case, it builds a manifest
list which contains images for all specified architectures. When you use this
image in
docker run
or
docker service
, Docker picks
the correct image based on the nodeâs platform.
You can build multi-platform images using three different strategies that are supported by Buildx and Dockerfiles:
QEMU is the easiest way to get started if your node already supports it (for
example. if you are using Docker Desktop). It requires no changes to your
Dockerfile and BuildKit automatically detects the secondary architectures that
are available. When BuildKit needs to run a binary for a different architecture,
it automatically loads it through a binary registered in the
binfmt_misc
handler.
For QEMU binaries registered with
binfmt_misc
on the host OS to work
transparently inside containers, they must be statically compiled and registered
with the
fix_binary
flag. This requires a kernel >= 4.8 and
binfmt-support >= 2.1.7. You can check for proper registration by checking if
F
is among the flags in
/proc/sys/fs/binfmt_misc/qemu-*
. While Docker
Desktop comes preconfigured with
binfmt_misc
support for additional platforms,
for other installations it likely needs to be installed using
tonistiigi/binfmt
image.
$ docker run --privileged --rm tonistiigi/binfmt --install all
Using multiple native nodes provide better support for more complicated cases
that are not handled by QEMU and generally have better performance. You can
add additional nodes to the builder instance using the
--append
flag.
Assuming contexts
node-amd64
and
node-arm64
exist in
docker context ls
;
$ docker buildx create --use --name mybuild node-amd64
mybuild
$ docker buildx create --append --name mybuild node-arm64
$ docker buildx build --platform linux/amd64,linux/arm64 .
Finally, depending on your project, the language that you use may have good
support for cross-compilation. In that case, multi-stage builds in Dockerfiles
can be effectively used to build binaries for the platform specified with
--platform
using the native architecture of the build node. A list of build
arguments like
BUILDPLATFORM
and
TARGETPLATFORM
is available automatically
inside your Dockerfile and can be leveraged by the processes running as part
of your build.
# syntax=docker/dockerfile:1
FROM --platform=$BUILDPLATFORM golang:alpine AS build
ARG TARGETPLATFORM
ARG BUILDPLATFORM
RUN echo "I am running on $BUILDPLATFORM, building for $TARGETPLATFORM" > /log
FROM alpine
COPY --from=build /log /log
Run the
docker buildx ls
command
to list the existing builders:
$ docker buildx ls
NAME/NODE DRIVER/ENDPOINT STATUS BUILDKIT PLATFORMS
default * docker
default default running 20.10.17 linux/amd64, linux/arm64, linux/arm/v7, linux/arm/v6
This displays the default builtin driver, that uses the BuildKit server
components built directly into the docker engine, also known as the
docker
driver.
Create a new builder using the
docker-container
driver
which gives you access to more complex features like multi-platform builds
and the more advanced cache exporters, which are currently unsupported in the
default
docker
driver:
$ docker buildx create --name mybuilder --driver docker-container --bootstrap
mybuilder
Switch to the new builder:
$ docker buildx use mybuilder
Note
Alternatively, run
docker buildx create --name mybuilder --driver docker-container --bootstrap --use
to create a new builder and switch to it using a single command.
And inspect it:
$ docker buildx inspect
Name: mybuilder
Driver: docker-container
Nodes:
Name: mybuilder0
Endpoint: unix:///var/run/docker.sock
Status: running
Buildkit: v0.10.4
Platforms: linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
Now listing the existing builders again, we can see our new builder is registered:
$ docker buildx ls
NAME/NODE DRIVER/ENDPOINT STATUS BUILDKIT PLATFORMS
mybuilder docker-container
mybuilder0 unix:///var/run/docker.sock running v0.10.4 linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
default * docker
default default running 20.10.17 linux/amd64, linux/arm64, linux/arm/v7, linux/arm/v6
Test the workflow to ensure you can build, push, and run multi-platform images. Create a simple example Dockerfile, build a couple of image variants, and push them to Docker Hub.
The following example uses a single
Dockerfile
to build an Alpine image with
cURL installed for multiple architectures:
# syntax=docker/dockerfile:1
FROM alpine:3.16
RUN apk add curl
Build the Dockerfile with buildx, passing the list of architectures to build for:
$ docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 -t <username>/<image>:latest --push .
...
#16 exporting to image
#16 exporting layers
#16 exporting layers 0.5s done
#16 exporting manifest sha256:71d7ecf3cd12d9a99e73ef448bf63ae12751fe3a436a007cb0969f0dc4184c8c 0.0s done
#16 exporting config sha256:a26f329a501da9e07dd9cffd9623e49229c3bb67939775f936a0eb3059a3d045 0.0s done
#16 exporting manifest sha256:5ba4ceea65579fdd1181dfa103cc437d8e19d87239683cf5040e633211387ccf 0.0s done
#16 exporting config sha256:9fcc6de03066ac1482b830d5dd7395da781bb69fe8f9873e7f9b456d29a9517c 0.0s done
#16 exporting manifest sha256:29666fb23261b1f77ca284b69f9212d69fe5b517392dbdd4870391b7defcc116 0.0s done
#16 exporting config sha256:92cbd688027227473d76e705c32f2abc18569c5cfabd00addd2071e91473b2e4 0.0s done
#16 exporting manifest list sha256:f3b552e65508d9203b46db507bb121f1b644e53a22f851185d8e53d873417c48 0.0s done
#16 ...
#17 [auth] <username>/<image>:pull,push token for registry-1.docker.io
#17 DONE 0.0s
#16 exporting to image
#16 pushing layers
#16 pushing layers 3.6s done
#16 pushing manifest for docker.io/<username>/<image>:latest@sha256:f3b552e65508d9203b46db507bb121f1b644e53a22f851185d8e53d873417c48
#16 pushing manifest for docker.io/<username>/<image>:latest@sha256:f3b552e65508d9203b46db507bb121f1b644e53a22f851185d8e53d873417c48 1.4s done
#16 DONE 5.6s
Note
<username>
must be a valid Docker ID and<image>
and valid repository on Docker Hub.- The
--platform
flag informs buildx to create Linux images for AMD 64-bit, Arm 64-bit, and Armv7 architectures.- The
--push
flag generates a multi-arch manifest and pushes all the images to Docker Hub.
Inspect the image using
docker buildx imagetools
command:
$ docker buildx imagetools inspect <username>/<image>:latest
Name: docker.io/<username>/<image>:latest
MediaType: application/vnd.docker.distribution.manifest.list.v2+json
Digest: sha256:f3b552e65508d9203b46db507bb121f1b644e53a22f851185d8e53d873417c48
Manifests:
Name: docker.io/<username>/<image>:latest@sha256:71d7ecf3cd12d9a99e73ef448bf63ae12751fe3a436a007cb0969f0dc4184c8c
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/amd64
Name: docker.io/<username>/<image>:latest@sha256:5ba4ceea65579fdd1181dfa103cc437d8e19d87239683cf5040e633211387ccf
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/arm64
Name: docker.io/<username>/<image>:latest@sha256:29666fb23261b1f77ca284b69f9212d69fe5b517392dbdd4870391b7defcc116
MediaType: application/vnd.docker.distribution.manifest.v2+json
Platform: linux/arm/v7
The image is now available on Docker Hub with the tag
<username>/<image>:latest
.
You can use this image to run a container on Intel laptops, Amazon EC2 Graviton
instances, Raspberry Pis, and on other architectures. Docker pulls the correct
image for the current architecture, so Raspberry PIs run the 32-bit Arm version
and EC2 Graviton instances run 64-bit Arm.
The digest identifies a fully qualified image variant. You can also run images targeted for a different architecture on Docker Desktop. For example, when you run the following on a macOS:
$ docker run --rm docker.io/<username>/<image>:latest@sha256:2b77acdfea5dc5baa489ffab2a0b4a387666d1d526490e31845eb64e3e73ed20 uname -m
aarch64
$ docker run --rm docker.io/<username>/<image>:latest@sha256:723c22f366ae44e419d12706453a544ae92711ae52f510e226f6467d8228d191 uname -m
armv7l
In the above example,
uname -m
returns
aarch64
and
armv7l
as expected,
even when running the commands on a native macOS or Windows developer machine.
Docker Desktop provides
binfmt_misc
multi-architecture support, which means you can run containers for different
Linux architectures such as
arm
,
mips
,
ppc64le
, and even
s390x
.
This does not require any special configuration in the container itself as it
uses qemu-static
from the
Docker for Mac VM
. Because of this, you can run an ARM container,
like the
arm32v7
or
ppc64le
variants of the busybox image.
Multi-stage builds are useful to anyone who has struggled to optimize Dockerfiles while keeping them easy to read and maintain.
Acknowledgment
Special thanks to Alex Ellis for granting permission to use his blog post Builder pattern vs. Multi-stage builds in Docker as the basis of the examples below.
One of the most challenging things about building images is keeping the image
size down. Each
RUN
,
COPY
, and
ADD
instruction in the Dockerfile adds a layer to the image, and you
need to remember to clean up any artifacts you donât need before moving on to
the next layer. To write a really efficient Dockerfile, you have traditionally
needed to employ shell tricks and other logic to keep the layers as small as
possible and to ensure that each layer has the artifacts it needs from the
previous layer and nothing else.
It was actually very common to have one Dockerfile to use for development (which contained everything needed to build your application), and a slimmed-down one to use for production, which only contained your application and exactly what was needed to run it. This has been referred to as the âbuilder patternâ. Maintaining two Dockerfiles is not ideal.
Hereâs an example of a
build.Dockerfile
and
Dockerfile
which adhere to the
builder pattern above:
build.Dockerfile
:
# syntax=docker/dockerfile:1
FROM golang:1.16
WORKDIR /go/src/github.com/alexellis/href-counter/
COPY app.go ./
RUN go get -d -v golang.org/x/net/html \
&& CGO_ENABLED=0 go build -a -installsuffix cgo -o app .
Notice that this example also artificially compresses two
RUN
commands together
using the Bash
&&
operator, to avoid creating an additional layer in the image.
This is failure-prone and hard to maintain. Itâs easy to insert another command
and forget to continue the line using the
\
character, for example.
Dockerfile
:
# syntax=docker/dockerfile:1
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY app ./
CMD ["./app"]
build.sh
:
#!/bin/sh
echo Building alexellis2/href-counter:build
docker build -t alexellis2/href-counter:build . -f build.Dockerfile
docker container create --name extract alexellis2/href-counter:build
docker container cp extract:/go/src/github.com/alexellis/href-counter/app ./app
docker container rm -f extract
echo Building alexellis2/href-counter:latest
docker build --no-cache -t alexellis2/href-counter:latest .
rm ./app
When you run the
build.sh
script, it needs to build the first image, create
a container from it to copy the artifact out, then build the second
image. Both images take up room on your system and you still have the
app
artifact on your local disk as well.
Multi-stage builds vastly simplify this situation!
With multi-stage builds, you use multiple
FROM
statements in your Dockerfile.
Each
FROM
instruction can use a different base, and each of them begins a new
stage of the build. You can selectively copy artifacts from one stage to
another, leaving behind everything you donât want in the final image. To show
how this works, letâs adapt the
Dockerfile
from the previous section to use
multi-stage builds.
# syntax=docker/dockerfile:1
FROM golang:1.16
WORKDIR /go/src/github.com/alexellis/href-counter/
RUN go get -d -v golang.org/x/net/html
COPY app.go ./
RUN CGO_ENABLED=0 go build -a -installsuffix cgo -o app .
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=0 /go/src/github.com/alexellis/href-counter/app ./
CMD ["./app"]
You only need the single Dockerfile. You donât need a separate build script,
either. Just run
docker build
.
$ docker build -t alexellis2/href-counter:latest .
The end result is the same tiny production image as before, with a significant reduction in complexity. You donât need to create any intermediate images, and you donât need to extract any artifacts to your local system at all.
How does it work? The second
FROM
instruction starts a new build stage with
the
alpine:latest
image as its base. The
COPY --from=0
line copies just the
built artifact from the previous stage into this new stage. The Go SDK and any
intermediate artifacts are left behind, and not saved in the final image.
By default, the stages are not named, and you refer to them by their integer
number, starting with 0 for the first
FROM
instruction. However, you can
name your stages, by adding an
AS <NAME>
to the
FROM
instruction. This
example improves the previous one by naming the stages and using the name in
the
COPY
instruction. This means that even if the instructions in your
Dockerfile are re-ordered later, the
COPY
doesnât break.
# syntax=docker/dockerfile:1
FROM golang:1.16 AS builder
WORKDIR /go/src/github.com/alexellis/href-counter/
RUN go get -d -v golang.org/x/net/html
COPY app.go ./
RUN CGO_ENABLED=0 go build -a -installsuffix cgo -o app .
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=builder /go/src/github.com/alexellis/href-counter/app ./
CMD ["./app"]
When you build your image, you donât necessarily need to build the entire
Dockerfile including every stage. You can specify a target build stage. The
following command assumes you are using the previous
Dockerfile
but stops at
the stage named
builder
:
$ docker build --target builder -t alexellis2/href-counter:latest .
A few scenarios where this might be very powerful are:
debug
stage with all debugging symbols or tools enabled, and a
lean
production
stage
testing
stage in which your app gets populated with test data, but
building for production using a different stage which uses real data
When using multi-stage builds, you are not limited to copying from stages you
created earlier in your Dockerfile. You can use the
COPY --from
instruction to
copy from a separate image, either using the local image name, a tag available
locally or on a Docker registry, or a tag ID. The Docker client pulls the image
if necessary and copies the artifact from there. The syntax is:
COPY --from=nginx:latest /etc/nginx/nginx.conf /nginx.conf
You can pick up where a previous stage left off by referring to it when using
the
FROM
directive. For example:
# syntax=docker/dockerfile:1
FROM alpine:latest AS builder
RUN apk --no-cache add build-base
FROM builder AS build1
COPY source1.cpp source.cpp
RUN g++ -o /binary source.cpp
FROM builder AS build2
COPY source2.cpp source.cpp
RUN g++ -o /binary source.cpp
Multi-stage build syntax was introduced in Docker Engine 17.05.
The legacy Docker Engine builder processes all stages of a Dockerfile leading
up to the selected
--target
. It will build a stage even if the selected
target doesnât depend on that stage.
BuildKit only builds the stages that the target stage depends on.
For example, given the following Dockerfile:
# syntax=docker/dockerfile:1
FROM ubuntu AS base
RUN echo "base"
FROM base AS stage1
RUN echo "stage1"
FROM base AS stage2
RUN echo "stage2"
With BuildKit enabled, building the
stage2
target in this Dockerfile means only
base
and
stage2
are processed.
There is no dependency on
stage1
, so itâs skipped.
$ DOCKER_BUILDKIT=1 docker build --no-cache -f Dockerfile --target stage2 .
[+] Building 0.4s (7/7) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 36B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/ubuntu:latest 0.0s
=> CACHED [base 1/2] FROM docker.io/library/ubuntu 0.0s
=> [base 2/2] RUN echo "base" 0.1s
=> [stage2 1/1] RUN echo "stage2" 0.2s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:f55003b607cef37614f607f0728e6fd4d113a4bf7ef12210da338c716f2cfd15 0.0s
On the other hand, building the same target without BuildKit results in all stages being processed:
$ DOCKER_BUILDKIT=0 docker build --no-cache -f Dockerfile --target stage2 .
Sending build context to Docker daemon 219.1kB
Step 1/6 : FROM ubuntu AS base
---> a7870fd478f4
Step 2/6 : RUN echo "base"
---> Running in e850d0e42eca
base
Removing intermediate container e850d0e42eca
---> d9f69f23cac8
Step 3/6 : FROM base AS stage1
---> d9f69f23cac8
Step 4/6 : RUN echo "stage1"
---> Running in 758ba6c1a9a3
stage1
Removing intermediate container 758ba6c1a9a3
---> 396baa55b8c3
Step 5/6 : FROM base AS stage2
---> d9f69f23cac8
Step 6/6 : RUN echo "stage2"
---> Running in bbc025b93175
stage2
Removing intermediate container bbc025b93175
---> 09fc3770a9c4
Successfully built 09fc3770a9c4
stage1
gets executed when BuildKit is disabled, even if
stage2
does not
depend on it.