Mutagen provides integration with Docker® Compose, allowing you to automatically create Mutagen synchronization and forwarding sessions alongside your Compose-based services, volumes, and networks. Mutagen’s synchronization and forwarding sessions can be used replace bind mounts and exposed ports, respectively, allowing you to run your project on a cloud-based Docker host while still editing code and accessing your application locally.
The following sections provide detailed information about using Mutagen’s
support for Docker Compose, but the best way to experience it is to look at an
example project and try it for yourself. You can find
an example web application
that demonstrates using this integration in the Mutagen repository. Have a look
at the corresponding
Compose YAML file
to get an idea of what’s involved with integrating Mutagen, and try starting the
mutagen compose up.
Here are a few other examples to get you started:
- A simple data science environment that uses Mutagen Compose integration
- A port of the Example Voting App that has been modified to support Mutagen Compose integration
Mutagen’s integration with Docker Compose uses
in Compose YAML files to define synchronization and forwarding sessions that
target Docker volumes and networks, respectively. The format of these extension
fields is described below. Note that using top-level
extension fields requires a Compose file with version 3.7 or later.
Once you’ve added these fields, you can use the
mutagen compose command to start and operate your Compose
mutagen compose command is a wrapper around the
command that reads
x-mutagen configuration information and creates, updates,
and terminates Mutagen synchronization and forwarding sessions alongside your
Compose-based services, volumes, and networks.
To define synchronization and forwarding sessions, you just need to include an
x-mutagen field at the root of your Compose YAML with Mutagen configuration
and session definitions. Here’s an example of what such a YAML file might look
# Set the Compose format version version: "3.7" # Define networks networks: frontend: backend: # Define volumes volumes: code: # Define services services: database: ... networks: - backend web: ... volumes: - code:/code networks: - frontend - backend ... # Define synchronization and forwarding sessions x-mutagen: sync: defaults: ignore: vcs: true code: alpha: "." beta: "volume://code" forward: database: source: "tcp:localhost:5432" destination: "network://backend:tcp:database:5432" web: source: "tcp::8080" destination: "network://frontend:tcp:web:8080"
In this example, we define a Mutagen session to synchronize code from the Docker
Compose project directory to a shared volume that we mount into our application
container. We also define two Mutagen forwarding sessions: one that exposes the
database to our local system (so that you can access it via a local client for
development purposes) and one that exposes a web appplication on all interfaces
on our local system (so that you can test it locally and from other devices on
your local network). In this example, we use a separate volume to store code and
we create networks manually. While volumes are necessary for synchronization
targets, you don’t need to explicitly create networks as in this example—you can
default network by name. We’ve also avoided any major
synchronization configuration here, whereas in real-world usage you’ll likely
want to set up ignores and
conflict resolution behavior. See
for a more complete setup.
The format of the
x-mutagen field is an extension of the
global configuration format described in the introduction.
In addition to a
defaults key for synchronization and forwarding, it allows
you to define keys that refer to named sessions. Sessions defined in Compose
projects can also have endpoint-specific configuration options, which are
specified using the
configurationBeta keys for
synchronization sessions and
configurationDestination keys for forwarding sessions. The most general
x-mutagen configuration looks something like:
x-mutagen: sync: defaults: ...default synchronization configuration... example-sync-session: alpha: ... beta: ... ...configuration... configurationAlpha: ...alpha-specific configuration... configurationBeta: ...beta-specific configuration... ... forward: defaults: ...default forwarding configuration... example-forwarding-session: source: ... destination: ... ...configuration... configurationSource: ...source-specific configuration... configurationDestination: ...destination-specific configuration... ...
As with normal Compose projects, you can specify these configuration elements
across multiple YAML files. When merging YAML files, Mutagen will merge
x-mutagen fields at the level below the
forward keys (i.e.
sessions with different names will be combined into a set, while sessions with
the same name (and session type) found in different files will just use the last
definition). This also applies to
Mutagen’s Compose integration currently supports only a limited set of endpoint URL formats, though these should suffice for typical Compose usage. For synchronization sessions, one of the endpoint URLs must be a local URL and the other must be a volume URL. For forwarding, the source endpoint URL must be a local URL and the destination endpoint URL must be a network URL. Support for Mutagen’s other transports is under development.
Volume URLs are of the form:
<volume-name> component must refer to the name of a volume defined in the
<path> component is optional and should be an absolute path rooted at the
volume root. If the
<path> component is omitted, then the volume root is used
as the synchronization target.
Network URLs are of the form:
<network-name> component must refer to the name of a network defined in
the Compose YAML. The
<network-endpoint> component is described in the
forwarding documentation. Only
TCP-based network endpoints are currently supported (i.e those prefixed with
tcp6) and the host component of the endpoint should refer to
a service defined in the Compose YAML and attached to the specified network.
In order to use the extension fields added to your Compose YAML, you’ll need to
use Mutagen’s Docker Compose wrapper command,
mutagen compose. This command
behaves identically to the
docker-compose command and provides awareness of
the synchronization and forwarding sessions defined in
mutagen compose wrapper is designed to faithfully emulate the
docker-compose command in every respect (including configuration loading,
override files and YAML merging,
.env file support, commands, flags, and so
on). If you find any observable difference in behavior, please
open an issue. In fact,
mutagen compose uses
docker-compose under the hood for most of the heavy
lifting, but it adds hooks to the
commands to manage Mutagen synchronization and forwarding sessions. These hooks
behave as follows:
- When running
up(with or without any service specifications), Mutagen synchronization and forwarding sessions are checked against their definitions in the
x-mutagenextension fields and recreated or terminated if their configuration has been modified or deleted, respectively. Any newly created synchronization sessions are flushed (i.e. they have a synchronization cycle forced) before any containers are created, allowing containers to be sure that code already exists on any relevant volumes before they start.
- When running
downwithout any service specifications, Mutagen sessions are terminated before terminating services or other infrastructure.
- When running
pswithout any flags or service specifications, Mutagen session statuses for the project are printed after container statuses. You can use
docker-compose psdirectly to avoid this behavior.
- When using
stopwithout any service specifications, Mutagen sessions are paused for the project.
- When using
start(with or without any service specifications), Mutagen sessions are resumed for the project before any services are started.
In short: you can use your normal Docker Compose workflow and expect Mutagen to manage sessions accordingly.
Mutagen’s Compose support works by emulating Docker Compose’s configuration
loading and determining the volume and network dependencies of synchronization
and forwarding sessions. It then defines an additional sidecar service (called
mutagen) attached to those volumes and networks through which synchronization
and forwarding can be routed. All of the heavy lifting of creating volumes,
networks, and containers is still provided by Docker Compose. The
service service uses the
Mutagen sidecar image (source code
available here), which is essentially a
no-op entrypoint suitable for hosting Mutagen agents. Creation of this service
is fully automated and managed by the
mutagen compose up and
mutagen compose down commands, so there’s no need to define it as part of your
Compose YAML (though it will show up alongside other containers in your
root, so all files synchronized onto volumes will have their default ownership set to
root. You can use Mutagen's permission settings to control the target UID/GID for files created by Mutagen. You can see an example of this in the data science demo.
mutagen service and all synchronization and forwarding sessions are
started before any of the project’s services are created. Additionally, all
synchronization sessions are flushed (forced to complete an initial
synchronization cycle) before any services are started, so you can be sure that
any code or other files being synchronized will be available on the target
volumes before any of your containers start. The synchronization and forwarding
sessions operate using Mutagen’s
Docker container transport.
Docker Compose and Docker CLI search paths
Mutagen uses the first
docker-compose executable found in the user’s path.
This search strategy can be overridden by setting the
MUTAGEN_DOCKER_COMPOSE_PATH environment variable to a path containing the
docker-compose executable. This environment variable should be set for
mutagen compose commands, not the Mutagen daemon.
Because Mutagen’s Compose support also uses the Docker CLI, the
described in the Docker container transport documentation
also applies and can be overridden using the
variable. Unlike the
MUTAGEN_DOCKER_COMPOSE_PATH variable, the
MUTAGEN_DOCKER_PATH variable should be set for the Mutagen daemon and not the
mutagen compose command.
There are a few known limitations with Mutagen’s Compose support that shouldn’t restrict most use cases. These will be improved/fixed in future releases.
- Implicit Docker contexts set using
docker context usearen’t supported at the moment because Mutagen sessions don’t query the current context setting from the Docker context store, meaning that any context selected in this way won’t be persisted. Docker contexts specified using the
--contextflags are supported and will persist correctly, and host configuration can also still be managed using
DOCKER_HOSTand other environment variables. This is a temporary limitation that will be removed as development continues.
- Flags that only exist in a short form (e.g. the
exec) are missing from shell completion in
mutagen compose. This is due to a limitation in Cobra’s flag parsing library tracked by spf13/pflag#139. Using these flags is still fully supported—only shell completion is affected.
--skip-hostname-checkflag is not supported because it’s not supported by the
dockercommand that Mutagen uses for its Docker container transport.
- Only local and Compose-related URLs (i.e.
network://URLs) are currently supported by Mutagen’s Compose integration. This is a temporary limitation that will be removed as development continues.
- Reverse network forwarding is not currently supported by Mutagen’s Compose integration. This is a temporary limitation that should be removed as development continues. You can still manually set up reverse forwarding with Docker containers in your project if necessary.
- Only Linux/x86-64 containers are currently supported by Mutagen’s Compose integration. This is a temporary limitation that will be removed as development continues. Support for Windows containers and additional architectures is expected.
runcommand is not fully supported because it creates ephemeral containers that Mutagen can’t easily track. The
runcommand may work fine in certain cases and support for it should improve as development continues.
Windows and Chocolatey
If you’re using Chocolatey to install Docker Compose
on Windows, please be aware that Chocolatey generates shim executables that can
interfere with Mutagen’s wrapping. Fortunately, you can point Mutagen to the
unshimmed executables to avoid this breakage by setting the
MUTAGEN_DOCKER_COMPOSE_PATH environment variable. This setting should look
You may need to adjust this path depending on your Chocolatey installation. You
can also use the
%ChocolateyInstall% environment variable as part of this
definition, for example: