Mount Audit Reporter

The problem

Mount tables drift without failing. Consider a common case: a project share is meant to be mounted with hard, someone unmounts it and mounts it again with soft during an outage to get a hung shell back, and the outage ends with the soft mount still in place. No error appears when the option changes; it is one word in a line nobody reads. A scratch filesystem that loses nodev or noexec behaves the same way, and so does a second mount laid over a path, or over a directory above it, that an earlier check inspected. The host keeps working while it no longer matches what an administrator meant it to be.

Anatomy of a Failover makes the case for hard mounts on data that matters, and nfs(5) warns that a soft timeout can cause silent data corruption in some cases. Soft is still right where responsiveness matters more than the data, such as scratch space or the storage mount exerciser's canary mount. What matters is that the choice was made on purpose and stays the way it was made.

A contract, compared with history

The baseline drift reporter records the mount table every night and reports any change since the last accepted baseline. That answers "what changed?" This tool answers "does the mount table match the contract?" A mount that was wrong on the day a baseline was accepted is part of the baseline; against a manifest, it is a finding, however long it has been wrong.

The tool checks what its manifest declares and nothing else, in the mount namespace it runs in. A mount that appears at an undeclared path produces no finding here; that is the baseline reporter's job. And a container, or a service with a private mount namespace, may see a different mount table from an interactive shell on the same host, so a manifest should be generated and audited from the same place.

The manifest

# <mountpoint>  <fstype>  <required_csv>  [forbidden_csv]  [source]
/var/tmp/build  tmpfs  nodev,noexec,nosuid       -
/mnt/projects   nfs4   hard,vers=4.1,proto=tcp   -    filer01.example.net:/export/projects
/data/backups   cifs   ro                        rw   //backup01.example.net/archive

Each line declares a mountpoint, its filesystem type, the options that must be present, the options that must be absent, and, where given, the source it must come from. The tool checks the live mount table against that statement; whether the statement itself is right is a decision for whoever reviews the file. The NFS and SMB lines are illustrative, written from nfs(5) and mount.cifs(8) rather than from a real host.

Columns are separated by whitespace, so a mountpoint containing a space cannot be written in this format. A # starts a comment only at the start of a line or after whitespace, so /mnt/#1 and server#A are ordinary values. A mountpoint is matched exactly as the mount table spells it: a trailing slash is ignored, and a symlink is never followed.

The source column catches what options cannot. Every option can be right while the mount points at the wrong place, such as a disaster recovery copy mounted for a test and never moved back. It is compared as a plain string, so an address and a hostname for the same server count as different; resolving names would make the audit depend on DNS at audit time.

Options as findmnt reports them

Options are matched as exact tokens against the option string findmnt builds from the kernel's mount table for the current namespace. That string describes the live mount, and it can differ from the command that was typed. On a test host, tmpfs mounts read back like this:

The size was rewritten, the order changed, and rw and relatime appeared unrequested. Defaults shown only by absence, such as exec, never appear at all, so the way to require an executable mount is to forbid noexec. A manifest written from memory fails on a correct mount, so generate mode prints a candidate from a known-good host, with a header listing what to prune, including options that change on their own on network mounts, such as addr=, clientaddr=, and negotiated rsize= and wsize= values nobody set on purpose:

bash mount_audit_reporter.sh --generate > client_mounts.manifest

For NFS, requiring the safe option beats forbidding the unsafe one. nfs(5) documents three recovery modes: hard, soft, and softerr. A rule that forbids soft misses softerr. In Linux 6.18 the NFS client writes hard into the option string whenever neither of the others is set, so on such a kernel requiring hard catches both. This tool has not yet audited an NFS mount, so confirm that with generate mode on the host.

Stacked mounts

When a second mount is made on a path that already has one, both stay in the mount table and only the newer one is visible. The first version of this script read the first line findmnt printed for the path. On a test host, a tmpfs mounted with nodev was covered by a second tmpfs without it, and against a manifest requiring nodev that version printed MOUNT AUDIT OK for a mount nobody could see.

The current version ignores print order. proc_pid_mountinfo(5) gives the rule: a mount made on top of another records that mount as its parent, so of the mounts at one path the visible one is the one that is no other's parent. The stack itself is reported as a note, because stacks are often deliberate; the lab workstation had one, made by the system, on 2026-10-19.

MOUNT AUDIT: 1 finding(s) across 1 mount(s) (stack.manifest)
  - /srv/lab/stacked: required option missing: nodev
  note: /srv/lab/stacked: 2 mounts stacked; audited the visible one

The same man page adds a condition, and a reviewer of the previous version found the hole it describes: the top mount at a path is reachable only if no later mount covers a directory above it. Mount a tmpfs at /srv/lab, another at /srv/lab/data, then a third over /srv/lab. The table still lists /srv/lab/data with its options, but the path now leads into the third mount, which has no data directory at all. That version checked the listed mount and printed MOUNT AUDIT OK. This one climbs the parent IDs from each declared mount and reports the cover:

MOUNT AUDIT: 1 finding(s) across 1 mount(s) (hidden.manifest)
  - /srv/lab/data: hidden: a later mount at /srv/lab covers it

The lab workstation shows why both rules are needed together. WSL2 mounts /tmp/.X11-unix, then a tmpfs over /tmp, then /tmp/.X11-unix again, so the table lists two mounts at that path, neither the other's parent, and the first one unreachable. The audit drops the covered one and checks the second. A cover anywhere above a mount decides it, and the outermost cover is named, so the result is the same in any print order. If the table still cannot single out one visible mount at a path, that path is an error rather than a guess.

How it fails

It refuses a manifest it cannot trust. Every line is validated before anything is audited. A bad column count, a relative mountpoint, an empty option, a duplicate mountpoint, an option both required and forbidden, a carriage return from a Windows editor, or an empty manifest all stop the run with exit 2. An audit that checked nothing never reports success.

It reads the table once. Each run reads the whole mount table with a single findmnt call that names its columns, so every mount is judged against the same listing, and a missing mount is a path with no row. That listing is not an atomic snapshot: the kernel releases its lock between reads, so a mount or unmount made while findmnt reads can show in it partly. findmnt exits 1 both when nothing matches and when it fails; a full listing always has rows, so any non-zero exit, a row the script cannot parse, or parent IDs that loop or name more than one root stop the run with exit 2.

Exit codes. 0 means every expectation was met, 1 means findings, and 2 means the audit could not be completed or trusted. The tool never calls mount or umount. Its only write is a temporary file from mktemp holding findmnt's error output, removed when findmnt exits or, if the run is interrupted, by a trap; a SIGKILL can leave it behind.

Running it nightly. The README gives a one-line cron entry that runs the audit under timeout and chronic -v. A clean run stays quiet. A failing one prints its output and exit status, so even an audit stopped by timeout before it printed anything sends mail, where the host has cron mail set up. A note on a clean run goes unseen under cron; that is deliberate, since a stack alone breaks no contract, and the baseline reporter records the full table every night.

The core of the script

The visible-mount rule, exactly as it appears in the repository. The parent climb is mount_is_covered, and the single findmnt call and its error handling are run_findmnt and load_mount_table, all in the same file.

# find_visible_mount <mountpoint>
#
# Finds the mount a process actually reaches at <mountpoint>, using the
# table load_mount_table read. Sets visible_mount_id and live_mount_count
# (how many mounts the table lists at that exact path). Returns:
#   0  one visible mount
#   1  nothing is mounted at that path
#   2  the table cannot say which mount is visible (mount_audit_query_error)
#   3  every mount there is covered by a later mount over a directory
#      above it; the outermost covering path is in hidden_by_target
#
# proc_pid_mountinfo(5) gives both rules. A mount made on top of another
# records that mount as its parent, so of the mounts stacked at one path
# a candidate for the top is one that is no other's parent, whatever
# order findmnt prints them in. And a top is reachable only if no later
# mount covers a directory above it, which mount_is_covered checks. A
# path can hold more than one candidate: WSL2 mounts /tmp/.X11-unix,
# then a tmpfs over /tmp, then /tmp/.X11-unix again, so the first mount
# is still listed but unreachable. Covered candidates are dropped; one
# left is the visible mount, more than one is an error, never a guess.
find_visible_mount() {
    local wanted_target="${1}";
    local row_id covered_status;
    local first_hidden_by="";
    local -a member_ids=();
    local -A member_parents=();

    visible_mount_id="";
    hidden_by_target="";
    for row_id in ${mount_table_at_target[${wanted_target}]:-}; do
        member_ids+=("${row_id}");
        [ "${mount_table_parent[${row_id}]}" = "${row_id}" ] || member_parents["${mount_table_parent[${row_id}]}"]=1;
    done;
    live_mount_count="${#member_ids[@]}";
    [ "${live_mount_count}" -eq 0 ] && return 1;

    for row_id in "${member_ids[@]}"; do
        [ -n "${member_parents[${row_id}]:-}" ] && continue;
        mount_is_covered "${row_id}";
        covered_status="$?";
        if [ "${covered_status}" -eq 2 ]; then
            return 2;
        fi;
        if [ "${covered_status}" -eq 3 ]; then
            if [ -z "${first_hidden_by}" ] || [ "${#hidden_by_target}" -lt "${#first_hidden_by}" ]; then
                first_hidden_by="${hidden_by_target}";
            fi;
            continue;
        fi;
        if [ -n "${visible_mount_id}" ]; then
            visible_mount_id="";
            mount_audit_query_error="cannot tell which of ${live_mount_count} stacked mounts is visible";
            return 2;
        fi;
        visible_mount_id="${row_id}";
    done;
    hidden_by_target="${first_hidden_by}";
    if [ -n "${visible_mount_id}" ]; then
        return 0;
    fi;
    if [ -n "${hidden_by_target}" ]; then
        return 3;
    fi;
    mount_audit_query_error="cannot tell which of ${live_mount_count} stacked mounts is visible";
    return 2;
};

The full script, the example manifest, and the tests are in the repository.

Evidence

84 bats tests run against a stand-in findmnt that reads a fixture mount table and assigns parent IDs the way the kernel does, so every finding type, stacks printed in either order, hidden mounts, every manifest error, failing findmnt runs, and generate mode are tested without root or real NFS and SMB servers. Stand-ins for mount and umount prove neither is called, and a scratch temporary directory stays empty on the normal and failing paths. As a check on the tests themselves, 63 deliberate defects were put back into a copy of the script one at a time; the suite failed for every one. It passes under Bash 4.2 through 5.3: the bash-compat workflow builds each release from source, and the lab runs also used the workstation's own Bash 5.3.

The script has run against real mount tables in two places: my WSL2 workstation (20 ext4, tmpfs, and 9p mounts; util-linux 2.41) and a disposable test host (util-linux 2.39). On both, tmpfs mounts were arranged for the test, stacked on one path and hidden by a later mount above them. findmnt reported the parent IDs proc_pid_mountinfo(5) describes, the audit checked the upper mount of each stack, and it reported each hidden mount. No NFS or SMB mount has been audited yet. Two loopback NFS attempts on WSL2 failed to mount, with the cause still open; a separate NFS server in the lab is the next step.

Known limitations

Exact tokens age with the kernel: an upgrade that changes how options are reported produces findings until the manifest is regenerated and reviewed. The tool checks mount-table metadata, not whether a filesystem works; a matching mount can still be unreachable or stale. Paths managed by autofs are a poor fit, since the audit never touches a path and so never triggers an automount. Mountpoints are matched as the table spells them, so a path reached through a symlink reads as not mounted. It has been tested on util-linux 2.39 and 2.41; a findmnt that rejects any of the six columns it asks for stops the run with exit 2.