Mutagen Extension for Docker Desktop
If you regularly work with large codebases in
Docker Desktop via
bind mounts, then you may have
encountered performance limitations with more demanding development tools and
dynamic language runtimes. In particular, the system call performance of the
virtual filesystems (such as Virtiofs, gRPC FUSE, and osxfs) used to
emulate bind mounts in Docker Desktop may not be sufficient to match native
performance. While these virtual filesystems are great for quickly making host
files available to containers, they can also become a bottleneck for larger
codebases and complex operations. Even relatively small codebases that bind
mount large dependency trees (such as
node_modules or Composer directories)
can suffer reduced performance when building code or running applications.
One of Mutagen’s most common use cases is to create synchronized caches of macOS and Windows filesystem contents in Linux-native ext4 volumes inside the Docker Desktop VM. These caches can replace the virtual filesystems used to implement bind mounts in Docker Desktop, offering increased performance for operations targeting those bind mounts.
To assist users in replacing bind mounts with synchronized filesystem caches, Mutagen offers a Docker Desktop extension that automates this process. This extension hides the underlying synchronization details from users and provides transparent replacement of bind mounts, enabling significant performance improvements when using Docker Desktop with large codebases.
The extension can be installed from the Docker Desktop Extensions Marketplace.
The extension is self-contained and does not require Mutagen to be installed.
To use the extension, you’ll need to create a filesystem cache for one or more locations from which you typically bind mount code. Once a cache is created and initialized, you can make use of it by targeting a special Docker context. This section outlines these steps in detail. For a quick and concrete example, check out the demo below.
To create a new cache, click the “Create Cache” button and select a directory:
When choosing cache locations, you’re typically best served by picking either a single cache location corresponding to the repository for the project that you’re working on, or multiple cache locations corresponding to the individual bind mounts that you want to create. Mutagen will use the most specific cache available to replace a bind mount, so it’s okay if caches overlap.
After selecting the cache location(s), you’ll be asked to set the default ownership for the new cache’s files in the VM. The extension will only replace bind mounts with a cache when the ownership for that cache matches the user and group ID for the container requesting the bind mount.
Most development containers run as
root by default (with a user and group ID
0), in which case you can leave the ownership set to
containerized development tooling will try to match the host OS user/group ID,
in which case you can tell the extension to do the same. Finally, some
containers use a custom user/group ID (such as
1000 for some other non-
root user), in which case you can specify these
manually. Ownership user/group IDs must be specified numerically because
name-based specifications are not portable between container images.
docker run --rm --entrypoint id <image> to print out the IDs.
You can also provide a
user specification to this
command, which can be useful if you know the username for the container
entrypoint but not the numeric user ID. For example, if you want to run an image
www-data, you can use
docker run --user www-data --rm --entrypoint id httpd:alpine.
Caches take a few seconds to initialize as files are copied into the Docker
Desktop VM. During this time, they will show a
Caches are only utilized once they are ready. If a container is created with a bind mount specification that doesn’t have a corresponding cache available and ready, then that bind mount specification will be passed through unmodified.
The extension pane presents a list of caches, each with the origin of their cached contents, the owner user and group IDs, and any other relevant status information. To get more detailed information about a cache, you can click on it to expose a pane detailing the cache’s content and utilization:
When caches are in-use by containers, they will have an
IN USE indicator.
Caches that are in-use cannot be deleted. Information about which containers are
using a cache can be found in the detail pane:
Once a cache is no longer is use, you can delete it using the “Delete Cache” button:
Mutagen Docker context
In order to take advantage of bind mount replacement, you will need to use a
desktop-linux-mutagen. This context serves as middleware for the
default Docker Desktop context (
desktop-linux), intercepting and modifying
container creation requests to enable bind mount replacement.
You can use this context on a per-command basis using the
flags, for example:
docker --context=desktop-linux-mutagen container create ...
docker --context=desktop-linux-mutagen compose up ...
Only container creation requests need to be intercepted; all other requests are passed unmodified to the standard Docker Desktop context. This means that you can use the Mutagen context as your default context without any difference in your workflows. To set the Mutagen context as your global default, you can use:
docker context use desktop-linux-mutagen
DOCKER_HOST environment variable to target
the Mutagen Docker endpoint. You can use
docker context ls to determine the
endpoint on your system.
As an example, try using the extension with the example-voting-app project. This is a relatively small project and thus won’t benefit tremendously from Mutagen enhancement, but it illustrates the typical Mutagen extension workflow.
Start by cloning the repository:
git clone https://github.com/dockersamples/example-voting-app.git
Next, create a cache of the repository using the Mutagen extension:
This project uses containers that run as
root, so you can leave the default
Once the cache is ready, you should see the following:
Filesystem changes made to this location/cache on the host or in the VM’s containers will now propagate via bidirectional synchronization. Cached files will be stored on an ext4-backed filesystem inside the Docker Desktop VM. This can offer substantial performance gains over the virtual filesystems typically used for bind mounts, especially for very large codebases (such as those with tens or hundreds of thousands of files).
Finally, switch to the repository and start the project via the Mutagen context:
docker --context=desktop-linux-mutagen compose up
Once the project starts, you’ll see that the cache is in use by four of the project’s containers:
To stop the project from utilizing the caches, you can tear it down using Docker Compose:
docker compose down
In the teardown case, the Mutagen context isn’t strictly necessary since no containers are being created (though you can still use the Mutagen context for this purpose, and you will be if it’s set as your default Docker context). After the project’s containers are no longer using the cache, you can delete the example cache with the “Delete Cache” button:
In normal usage, you’d leave the cache active for a much longer period of time (typically however long you’re working on the corresponding project).
License and pricing
The Mutagen Extension for Docker Desktop is separate from Mutagen and Mutagen Compose. It is offered under its own End-User License Agreement.
The extension offers a free usage tier that includes one cache. Creating multiple caches and using custom permissions requires a Mutagen Pro subscription that can be purchased here.
The initial release of this extension is focused on replicating the behavior of Docker Desktop bind mounts with enhanced performance, but additional functionality is being developed. The development roadmap for the extension will be driven by user feedback, but currently planned additions include:
- Ad hoc temporary cache creation
- Custom ignores
- Remote Docker engine support
The extension has the following known limitations:
- The extension pane must be opened when Docker Desktop relaunches in order to trigger complete startup of the extension. This only needs to be done once after restarting Docker Desktop. Caches will continue to function even if the pane is not clicked, but synchronization cannot resume until the pane is activated.
- The data volume backing the extension is currently visible. This is managed by the extension commands and should not be managed or utilized directly by users.
Both of these issues will likely be addressable in the future as the extension platform and SDK evolve.
Conflicts and problems
As with all synchronization solutions, conflicts and problems can arise. These will be indicated on the corresponding cache pane:
Non-conflicting and non-problematic files will continue to synchronize even if conflicts or problems are present, so these indications are not catastrophic.
Mutagen uses heuristics to auto-resolve most conflicts, meaning that conflicts will typically be limited to the deletion of parent directories of content that isn’t normally synchronized (such as Unix domain sockets or FIFOs).
Problems, on the other hand, are typically due to an inability to scan, update, or delete files because of insufficient permissions.
In all cases, conflicts can be resolved by deleting the side of the conflict that should “lose” and problems can be resolved by fixing their corresponding issues.
In order to help us design better auto-resolution heuristics, please feel free to report and discuss conflicts and problems via the Mutagen Docker Desktop Extension issue tracker or the Mutagen Community Slack Workspace.