Mutagen Extension for Docker Desktop
If you regularly work with large codebases in
Docker Desktop via
bind mounts, then you may have
encountered performance limitations with more demanding development tools and
dynamic language runtimes. In particular, the system call performance of the
virtual filesystems (such as Virtiofs, gRPC FUSE, and osxfs) used to
emulate bind mounts in Docker Desktop may not be sufficient to match native
performance. While these virtual filesystems are great for quickly making host
files available to containers, they can also become a bottleneck for larger
codebases and complex operations. Even relatively small codebases that bind
mount large dependency trees (such as node_modules
or Composer directories)
can suffer reduced performance when building code or running applications.
One of Mutagen’s most common use cases is to create synchronized caches of macOS and Windows filesystem contents in Linux-native ext4 volumes inside the Docker Desktop VM. These caches can replace the virtual filesystems used to implement bind mounts in Docker Desktop, offering increased performance for operations targeting those bind mounts.
To assist users in replacing bind mounts with synchronized filesystem caches, Mutagen offers a Docker Desktop extension that automates this process. This extension hides the underlying synchronization details from users and provides transparent replacement of bind mounts, enabling significant performance improvements when using Docker Desktop with large codebases.
Installation
The extension can be installed from the Docker Desktop Extensions Marketplace.
The extension is self-contained and does not require Mutagen to be installed.
Usage
To use the extension, you’ll need to create a filesystem cache for one or more locations from which you typically bind mount code. Once a cache is created and initialized, you can make use of it by targeting a special Docker context. This section outlines these steps in detail. For a quick and concrete example, check out the demo below.
Cache creation
To create a new cache, click the “Create Cache” button and select a directory:
When choosing cache locations, you’re typically best served by picking either a single cache location corresponding to the repository for the project that you’re working on, or multiple cache locations corresponding to the individual bind mounts that you want to create. Mutagen will use the most specific cache available to replace a bind mount, so it’s okay if caches overlap.
Ownership
After selecting the cache location(s), you’ll be asked to set the default ownership for the new cache’s files in the VM. The extension will only replace bind mounts with a cache when the ownership for that cache matches the user and group ID for the container requesting the bind mount.
Most development containers run as root
by default (with a user and group ID
of 0
), in which case you can leave the ownership set to root
. Some
containerized development tooling will try to match the host OS user/group ID,
in which case you can tell the extension to do the same. Finally, some
containers use a custom user/group ID (such as 33
or 82
for www-data
, or
1000
for some other non-root
user), in which case you can specify these
manually. Ownership user/group IDs must be specified numerically because
name-based specifications are not portable between container images.
docker run --rm --entrypoint id <image>
to print out the IDs.
You can also provide a
user specification to this
command, which can be useful if you know the username for the container
entrypoint but not the numeric user ID. For example, if you want to run an image
based on httpd:alpine
as www-data
, you can use
docker run --user www-data --rm --entrypoint id httpd:alpine
.Initialization
Caches take a few seconds to initialize as files are copied into the Docker
Desktop VM. During this time, they will show a PREPARING
indicator.
Caches are only utilized once they are ready. If a container is created with a bind mount specification that doesn’t have a corresponding cache available and ready, then that bind mount specification will be passed through unmodified.
Inspecting caches
The extension pane presents a list of caches, each with the origin of their cached contents, the owner user and group IDs, and any other relevant status information. To get more detailed information about a cache, you can click on it to expose a pane detailing the cache’s content and utilization:
When caches are in-use by containers, they will have an IN USE
indicator.
Caches that are in-use cannot be deleted. Information about which containers are
using a cache can be found in the detail pane:
Deleting caches
Once a cache is no longer is use, you can delete it using the “Delete Cache” button:
Mutagen Docker context
In order to take advantage of bind mount replacement, you will need to use a
special
Docker Context
called desktop-linux-mutagen
. This context serves as middleware for the
default Docker Desktop context (desktop-linux
), intercepting and modifying
container creation requests to enable bind mount replacement.
You can use this context on a per-command basis using the -c
/--context
flags, for example:
docker --context=desktop-linux-mutagen container create ...
docker --context=desktop-linux-mutagen compose up ...
Only container creation requests need to be intercepted; all other requests are passed unmodified to the standard Docker Desktop context. This means that you can use the Mutagen context as your default context without any difference in your workflows. To set the Mutagen context as your global default, you can use:
docker context use desktop-linux-mutagen
DOCKER_HOST
environment variable to target
the Mutagen Docker endpoint. You can use docker context ls
to determine the
endpoint on your system.Example
As an example, try using the extension with the example-voting-app project. This is a relatively small project and thus won’t benefit tremendously from Mutagen enhancement, but it illustrates the typical Mutagen extension workflow.
Start by cloning the repository:
git clone https://github.com/dockersamples/example-voting-app.git
Next, create a cache of the repository using the Mutagen extension:
This project uses containers that run as root
, so you can leave the default
ownership setting:
Once the cache is ready, you should see the following:
Filesystem changes made to this location/cache on the host or in the VM’s containers will now propagate via bidirectional synchronization. Cached files will be stored on an ext4-backed filesystem inside the Docker Desktop VM. This can offer substantial performance gains over the virtual filesystems typically used for bind mounts, especially for very large codebases (such as those with tens or hundreds of thousands of files).
Finally, switch to the repository and start the project via the Mutagen context:
cd example-voting-app
docker --context=desktop-linux-mutagen compose up
Once the project starts, you’ll see that the cache is in use by four of the project’s containers:
To stop the project from utilizing the caches, you can tear it down using Docker Compose:
docker compose down
In the teardown case, the Mutagen context isn’t strictly necessary since no containers are being created (though you can still use the Mutagen context for this purpose, and you will be if it’s set as your default Docker context). After the project’s containers are no longer using the cache, you can delete the example cache with the “Delete Cache” button:
In normal usage, you’d leave the cache active for a much longer period of time (typically however long you’re working on the corresponding project).
License and pricing
The Mutagen Extension for Docker Desktop is separate from Mutagen and Mutagen Compose. It is offered under its own End-User License Agreement.
The extension offers a free usage tier that includes one cache. Creating multiple caches and using custom permissions requires a Mutagen Pro subscription that can be purchased here.
Roadmap
The initial release of this extension is focused on replicating the behavior of Docker Desktop bind mounts with enhanced performance, but additional functionality is being developed. The development roadmap for the extension will be driven by user feedback, but currently planned additions include:
- Ad hoc temporary cache creation
- Custom ignores
- Remote Docker engine support
Known limitations
The extension has the following known limitations:
- The extension pane must be opened when Docker Desktop relaunches in order to trigger complete startup of the extension. This only needs to be done once after restarting Docker Desktop. Caches will continue to function even if the pane is not clicked, but synchronization cannot resume until the pane is activated.
- The data volume backing the extension is currently visible. This is managed by the extension commands and should not be managed or utilized directly by users.
Both of these issues will likely be addressable in the future as the extension platform and SDK evolve.
Conflicts and problems
As with all synchronization solutions, conflicts and problems can arise. These will be indicated on the corresponding cache pane:
Non-conflicting and non-problematic files will continue to synchronize even if conflicts or problems are present, so these indications are not catastrophic.
Mutagen uses heuristics to auto-resolve most conflicts, meaning that conflicts will typically be limited to the deletion of parent directories of content that isn’t normally synchronized (such as Unix domain sockets or FIFOs).
Problems, on the other hand, are typically due to an inability to scan, update, or delete files because of insufficient permissions.
In all cases, conflicts can be resolved by deleting the side of the conflict that should “lose” and problems can be resolved by fixing their corresponding issues.
In order to help us design better auto-resolution heuristics, please feel free to report and discuss conflicts and problems via the Mutagen Docker Desktop Extension issue tracker or the Mutagen Community Slack Workspace.