Watching

Mutagen uses filesystem watching to know when it should scan for and propagate changes. Unfortunately, the filesystem watching landscape is extremely varied in terms of implementation, efficiency, and robustness. Almost every platform uses a completely different mechanism, many of which are unreliable or non-scalable.

Some systems (namely macOS and Windows) provide native recursive watching mechanisms that can monitor arbitrarily large directory hierarchies, though their behavior when the location being watched is deleted or changed is somewhat inconsistent, and Windows only supports using a directory as the root of such a recursive watch.

Other systems (for example, Linux^® and BSD systems) provide mechanisms that require a watch descriptor or file descriptor to be open for every file or directory being watched in a directory hierarchy, which can quickly exhaust system quotas for directories that might be used in development (for example, imagine a synchronization root containing a node_modules directory).

Mutagen takes a pragmatic approach to filesystem watching that attempts to maximize reliability and responsiveness while avoiding exhaustion of system resources or problematic behavior.

Native recursive watching

On systems that natively support recursive filesystem watching, a watch is established on either the synchronization root itself (on macOS) or the parent directory of the synchronization root (on Windows) and events are filtered to only those originating from the synchronization root. Because these systems can behave strangely if the root of a watch is deleted, a regular (but very cheap) polling mechanism is used to ensure that the watch root hasn’t been deleted or recreated. If a change to the watch root is detected, the watch is re-established.

Polling

On all other systems, a polling mechanism is used to avoid exhausting watch/file descriptors. This polling mechanism is also used as a fallback in cases where native watching mechanisms fail unrecoverably. On Linux, this polling is coupled with a restricted set of native watches on the most recently updated contents, allowing for low-latency change notifications for most workflows without exhausting watch/file descriptors.

Modes

Mutagen provides three different filesystem watching modes:

portable (Default): In this mode, Mutagen uses the most efficient watching implementation possible for an endpoint. If some type of native watching mechanism is available, it is used, otherwise pure poll-based watching is used.
force-poll: In this mode, Mutagen will always use its poll-based watching implementation, even on systems that support native watching.
no-watch: In this mode, Mutagen won’t perform any filesystem watching, and endpoints using this mode will not be able to trigger a synchronization cycle. If this mode is used for the entire synchronization session, then a synchronization cycle only occurs when the mutagen sync flush command is used.

These modes can be specified on a per-session and/or per-endpoint basis by passing the --watch-mode=<mode> or --watch-mode-(alpha|beta)=<mode> flag, respectively, to the mutagen sync create command. These modes can be specified on a default per-session basis by including the following configuration in ~/.mutagen.yml:

sync:
  defaults:
    watch:
      mode: "<mode>"

Polling interval

If using polling-based watching, the polling interval (which defaults to 10 seconds) can be specified on a per-session and/or per-endpoint basis by passing the --watch-polling-interval=<interval> or --watch-polling-interval-(alpha|beta)=<interval> flag, respectively, to the mutagen sync create command (where <interval> is an integer value representing seconds) and on a default per-session basis by including the following configuration in ~/.mutagen.yml:

sync:
  defaults:
    watch:
      pollingInterval: <interval>