Compose V1 Integration (Deprecated)
This documentation is provided as a reference for users who have not yet migrated to the new Mutagen Compose V2.
Mutagen provides integration with Docker® Compose, allowing you to automatically create Mutagen synchronization and forwarding sessions alongside your Compose-based services, volumes, and networks. Mutagen’s synchronization and forwarding sessions can be used replace bind mounts and exposed ports, respectively, allowing you to run your project on a cloud-based Docker host while still editing code and accessing your application locally.
The following sections provide detailed information about using Mutagen’s
support for Docker Compose, but the best way to experience it is to look at an
example project and try it for yourself. You can find
an example web application
that demonstrates using this integration in the Mutagen repository. Have a look
at the corresponding
Compose YAML file
to get an idea of what’s involved with integrating Mutagen, and try starting the
mutagen compose up.
Here are a few other examples to get you started:
- A simple data science environment that uses Mutagen Compose integration
- A port of the Example Voting App that has been modified to support Mutagen Compose integration
Mutagen’s integration with Docker Compose uses
in Compose YAML files to define synchronization and forwarding sessions that
target Docker volumes and networks, respectively. The format of these extension
fields is described below. Note that using top-level
extension fields requires a Compose file with version 3.7 or later.
Once you’ve added these fields, you can use the
mutagen compose command to start and operate your Compose
mutagen compose command is a wrapper around the
command that reads
x-mutagen configuration information and creates, updates,
and terminates Mutagen synchronization and forwarding sessions alongside your
Compose-based services, volumes, and networks.
To define synchronization and forwarding sessions, you just need to include an
x-mutagen field at the root of your Compose YAML with Mutagen configuration
and session definitions. Here’s an example of what such a YAML file might look
# Set the Compose format version version: "3.7" # Define networks networks: frontend: backend: # Define volumes volumes: code: # Define services services: database: ... networks: - backend web: ... volumes: - code:/code networks: - frontend - backend ... # Define synchronization and forwarding sessions x-mutagen: sync: defaults: ignore: vcs: true code: alpha: "." beta: "volume://code" forward: database: source: "tcp:localhost:5432" destination: "network://backend:tcp:database:5432" web: source: "tcp::8080" destination: "network://frontend:tcp:web:8080"
In this example, we define a Mutagen session to synchronize code from the Docker
Compose project directory to a shared volume that we mount into our application
container. We also define two Mutagen forwarding sessions: one that exposes the
database to our local system (so that you can access it via a local client for
development purposes) and one that exposes a web appplication on all interfaces
on our local system (so that you can test it locally and from other devices on
your local network). In this example, we use a separate volume to store code and
we create networks manually. While volumes are necessary for synchronization
targets, you don’t need to explicitly create networks as in this example—you can
default network by name. We’ve also avoided any major
synchronization configuration here, whereas in real-world usage you’ll likely
want to set up ignores and
conflict resolution behavior. See
for a more complete setup.
The format of the
x-mutagen field is an extension of the
global configuration format described in the introduction.
In addition to a
defaults key for synchronization and forwarding, it allows
you to define keys that refer to named sessions. Sessions defined in Compose
projects can also have endpoint-specific configuration options, which are
specified using the
configurationBeta keys for
synchronization sessions and
configurationDestination keys for forwarding sessions. The most general
x-mutagen configuration looks something like:
x-mutagen: sync: defaults: ...default synchronization configuration... example-sync-session: alpha: ... beta: ... ...configuration... configurationAlpha: ...alpha-specific configuration... configurationBeta: ...beta-specific configuration... ... forward: defaults: ...default forwarding configuration... example-forwarding-session: source: ... destination: ... ...configuration... configurationSource: ...source-specific configuration... configurationDestination: ...destination-specific configuration... ...
As with normal Compose projects, you can specify these configuration elements
across multiple YAML files. When merging YAML files, Mutagen will merge
x-mutagen fields at the level below the
forward keys (i.e.
sessions with different names will be combined into a set, while sessions with
the same name (and session type) found in different files will just use the last
definition). This also applies to
Mutagen’s Compose V1 integration supports only a limited set of endpoint URL formats, though these should suffice for typical Compose usage. For synchronization sessions, one of the endpoint URLs must be a local URL and the other must be a volume URL. For forwarding, the source endpoint URL must be a local URL and the destination endpoint URL must be a network URL. Support for Mutagen’s other transports is under development.
Volume URLs are of the form:
<volume-name> component must refer to the name of a volume defined in the
<path> component is optional and should be an absolute path rooted at the
volume root. If the
<path> component is omitted, then the volume root is used
as the synchronization target.
Network URLs are of the form:
<network-name> component must refer to the name of a network defined in
the Compose YAML. The
<network-endpoint> component is described in the
forwarding documentation. Only
TCP-based network endpoints are currently supported (i.e those prefixed with
tcp6) and the host component of the endpoint should refer to
a service defined in the Compose YAML and attached to the specified network.
In order to use the extension fields added to your Compose YAML, you’ll need to
use Mutagen’s Docker Compose wrapper command,
mutagen compose. This command
behaves identically to the
docker-compose command and provides awareness of
the synchronization and forwarding sessions defined in
mutagen compose wrapper is designed to faithfully emulate the
docker-compose command in every respect (including configuration loading,
override files and YAML merging,
.env file support, commands, flags, and so
on). If you find any observable difference in behavior, please
open an issue. In fact,
mutagen compose uses
docker-compose under the hood for most of the heavy
lifting, but it adds hooks to the
commands to manage Mutagen synchronization and forwarding sessions. These hooks
behave as follows:
- When running
up(with or without any service specifications), Mutagen synchronization and forwarding sessions are checked against their definitions in the
x-mutagenextension fields and recreated or terminated if their configuration has been modified or deleted, respectively. Any newly created synchronization sessions are flushed (i.e. they have a synchronization cycle forced) before any containers are created, allowing containers to be sure that code already exists on any relevant volumes before they start.
- When running
downwithout any service specifications, Mutagen sessions are terminated before terminating services or other infrastructure.
- When running
pswithout any flags or service specifications, Mutagen session statuses for the project are printed after container statuses. You can use
docker-compose psdirectly to avoid this behavior.
- When using
stopwithout any service specifications, Mutagen sessions are paused for the project.
- When using
start(with or without any service specifications), Mutagen sessions are resumed for the project before any services are started.
In short: you can use your normal Docker Compose workflow and expect Mutagen to manage sessions accordingly.
Mutagen’s Compose V1 support works by emulating Docker Compose’s configuration
loading and determining the volume and network dependencies of synchronization
and forwarding sessions. It then defines an additional sidecar service (called
mutagen) attached to those volumes and networks through which synchronization
and forwarding can be routed. All of the heavy lifting of creating volumes,
networks, and containers is still provided by Docker Compose. The
service service uses the
Mutagen sidecar image (source code
available here), which is essentially a
no-op entrypoint suitable for hosting Mutagen agents. Creation of this service
is fully automated and managed by the
mutagen compose up and
mutagen compose down commands, so there’s no need to define it as part of your
Compose YAML (though it will show up alongside other containers in your
root, so all files synchronized onto volumes will have their default ownership set to
root. You can use Mutagen's permission settings to control the target UID/GID for files created by Mutagen. You can see an example of this in the data science demo.
mutagen service and all synchronization and forwarding sessions are
started before any of the project’s services are created. Additionally, all
synchronization sessions are flushed (forced to complete an initial
synchronization cycle) before any services are started, so you can be sure that
any code or other files being synchronized will be available on the target
volumes before any of your containers start. The synchronization and forwarding
sessions operate using Mutagen’s
Docker container transport.
In Mutagen v0.13.0-beta2 and later, the Mutagen sidecar image also causes a few minor behavioral changes in Mutagen agents that are designed to make working with Compose projects easier. Specifically:
- If a synchronization session targets an empty volume root and a non-default owner, group, or directory mode setting has been specified, then Mutagen will set the permissions of the volume root to match. This is a heuristic to work around the fact that Docker volumes don’t have an easy way to set permissions upon creation.
- If no file staging mode is explicitly set in the container, then Mutagen will
will stage files into a temporary directory in their target volumes, avoiding
the need for file copy operations when applying changes. If this behavior is
undesirable (e.g. due to conflicts with filesystem watchers), then you can
choose to stage in the sidecar container filesystem itself by selecting the
Docker Compose and Docker CLI search paths
Mutagen uses the first
docker-compose executable found in the user’s path.
This search strategy can be overridden by setting the
MUTAGEN_DOCKER_COMPOSE_PATH environment variable to a path containing the
docker-compose executable. This environment variable should be set for
mutagen compose commands, not the Mutagen daemon.
Because Mutagen’s Compose support also uses the Docker CLI, the
described in the Docker container transport documentation
also applies and can be overridden using the
variable. Unlike the
MUTAGEN_DOCKER_COMPOSE_PATH variable, the
MUTAGEN_DOCKER_PATH variable should be set for the Mutagen daemon and not the
mutagen compose command.
There are a few known limitations with Mutagen’s Compose V1 support that shouldn’t restrict most use cases:
- Implicit Docker contexts set using
docker context usearen’t supported because Mutagen sessions don’t query the current context setting from the Docker context store, meaning that any context selected in this way won’t be persisted. Docker contexts specified using the
--contextflags are supported and will persist correctly, and host configuration can also still be managed using
DOCKER_HOSTand other environment variables.
- Restarting the target Docker daemon in Docker Desktop will result in Mutagen sessions being duplicated due to the Docker daemon identifier changing.
- Flags that only exist in a short form (e.g. the
exec) are missing from shell completion in
mutagen compose. This is due to a limitation in Cobra’s flag parsing library tracked by spf13/pflag#139. Using these flags is still fully supported—only shell completion is affected.
--skip-hostname-checkflag is not supported because it’s not supported by the
dockercommand that Mutagen uses for its Docker container transport.
- Only local and Compose-related URLs (i.e.
network://URLs) are supported by Mutagen’s Compose V1 integration.
- Reverse network forwarding is not supported by Mutagen’s Compose V1 integration. You can still manually set up reverse forwarding with Docker containers in your project if necessary.
- Only Linux containers are supported by Mutagen’s Compose V1 integration.
runcommand is not fully supported because it creates ephemeral containers that Mutagen can’t easily track.
Windows and Chocolatey
If you’re using Chocolatey to install Docker Compose
on Windows, please be aware that Chocolatey generates shim executables that can
interfere with Mutagen’s wrapping. Fortunately, you can point Mutagen to the
unshimmed executables to avoid this breakage by setting the
MUTAGEN_DOCKER_COMPOSE_PATH environment variable. This setting should look
You may need to adjust this path depending on your Chocolatey installation. You
can also use the
%ChocolateyInstall% environment variable as part of this
definition, for example: