[{"content":"Introduction On a set of shared, multi-user Linux workstations, several things broke in the same few weeks. Files on a user\u0026rsquo;s NFS home showed up as nobody:nogroup. Once we tightened the share, a user could no longer write to their own home, even though ls showed them as the owner. A freshly provisioned home dataset came out owned by root:root with mode 755 instead of belonging to the user with mode 700. A user\u0026rsquo;s Jupyter container opened with the prompt I have no name! and sudo refused to work.\nMost of this came from one root cause. The same Active Directory user had one UID on the Linux clients and a different UID on the storage server. Neither side was misconfigured on its own terms. In the end, we computed the base the clients used and set the storage server\u0026rsquo;s ID range to start at the same number. Two more lookups on the same path, one on the client and one on the server, also had to be fixed before access worked.\nThis post follows one identity from login to an NFS file, shows where the mismatch hurts, and covers the fix, a checklist, and alternatives.\nThe Setup The environment has four parts:\nDomain controller: a Samba-based AD appliance for the EXAMPLE.LOCAL realm. Storage: TrueNAS SCALE at nas.example.local (192.0.2.10), joined to the same domain. Every user gets a separate ZFS dataset, exported over NFSv4 with sec=krb5 only. Each dataset is owned by its user with mode 0700. The owner has full access and everyone else has none. Clients: Ubuntu workstations joined to the domain with SSSD. pam_mount mounts the user\u0026rsquo;s dataset at login, and rpc.gssd handles the Kerberos side of NFS. Docker: users run their work (Jupyter, training jobs) in containers. The user\u0026rsquo;s NFS directory is bind-mounted into the container as /workspace. The isolation goal is simple: user A must not be able to read user B\u0026rsquo;s dataset, even with Docker in the picture.\nThe diagram shows which identity is in effect at each hop, before the fix. The UIDs are examples.\nflowchart TD A[\u0026#34;Login on client\u0026lt;br/\u0026gt;AD password\u0026#34;] --\u0026gt; B[\u0026#34;SSSD\u0026lt;br/\u0026gt;SID to UID (ID mapping)\u0026lt;br/\u0026gt;alice = 1914801105\u0026#34;] B --\u0026gt; C[\u0026#34;SSSD krb5_child\u0026lt;br/\u0026gt;TGT in /tmp/krb5cc_1914801105\u0026#34;] C --\u0026gt; D[\u0026#34;pam_mount\u0026lt;br/\u0026gt;mount nas:/.../alice sec=krb5\u0026#34;] D --\u0026gt; E[\u0026#34;docker exec --user 1914801105\u0026lt;br/\u0026gt;process in container\u0026#34;] E --\u0026gt; F[\u0026#34;Kernel NFS client\u0026lt;br/\u0026gt;caller UID = 1914801105\u0026#34;] F --\u0026gt; G[\u0026#34;rpc.gssd\u0026lt;br/\u0026gt;finds ccache by UID\u0026lt;br/\u0026gt;principal alice@EXAMPLE.LOCAL\u0026#34;] G --\u0026gt; H[\u0026#34;TrueNAS\u0026lt;br/\u0026gt;principal to UID via its own idmap (RID)\u0026lt;br/\u0026gt;alice = 100001106\u0026#34;] H --\u0026gt; I[\u0026#34;ZFS permission check\u0026lt;br/\u0026gt;owner 100001106, mode 0700\u0026#34;] I --\u0026gt; J[\u0026#34;Reply: owner as name string\u0026lt;br/\u0026gt;EXAMPLE\\\\alice@example.local\u0026#34;] J --\u0026gt; K[\u0026#34;Client nfsidmap + SSSD\u0026lt;br/\u0026gt;name to local UID 1914801105\u0026#34;] The client\u0026rsquo;s number never reaches the server; it only selects the Kerberos ticket. The server uses the principal and its own number, and the owner comes back as a name. The two numbers never meet, which is why the mismatch could hide.\nWhy Containers Must Not Run as Root Here With sec=sys, the server trusts the UID the client sends. With sec=krb5, it trusts a Kerberos principal, and the client picks that principal from the UID of the calling process.\nHow the client picks a Kerberos identity Per the rpc.gssd man page, user credentials live in a file with \u0026ldquo;a well-known name constructed using the user\u0026rsquo;s UID\u0026rdquo;, typically in /tmp. SSSD writes the ticket cache under exactly that kind of name:\n# /etc/sssd/sssd.conf (domain section, excerpt) ldap_id_mapping = True krb5_ccachedir = /tmp krb5_ccname_template = FILE:%d/krb5cc_%U krb5_renewable_lifetime = 7d krb5_renew_interval = 60m %U is the login UID and %d is krb5_ccachedir (sssd-krb5(5)). Renewal keeps a long Jupyter session from losing its ticket.\nA container shares the host kernel, so when a container process touches /workspace, rpc.gssd looks for a ticket cache under that process\u0026rsquo;s UID. The container\u0026rsquo;s UID decides which identity the NAS sees.\nWhat root gets The rpc.gssd man page is explicit about root: \u0026ldquo;By default rpc.gssd uses the system\u0026rsquo;s machine credentials for UID 0 accesses that require GSS authentication.\u0026rdquo;\nSo root in the container reaches the NAS as the machine account, which owns none of the 0700 datasets. Root gets less access to the user\u0026rsquo;s files than the user. That is good for security, but any setup step that writes to /workspace as root fails.\nIt failed even earlier for us. The launcher started containers as root with -w /workspace. After the share became Kerberos-only, the container no longer started at all: the runtime\u0026rsquo;s chdir into /workspace ran as root and was denied. The fix was to start the container with -w /, so root never touches the NFS path, and to change into /workspace only in the exec calls that run as the user.\nOur launcher used to create Jupyter directories as root and then chown -R them to the user. On a Kerberos mount that step cannot work. We replaced it with a mkdir run as the user:\n# Create directories AS THE USER (NFS, owner is the user) docker exec --user \u0026#34;$UID:$GID\u0026#34; \u0026#34;$name\u0026#34; bash -c \u0026#39; mkdir -p /workspace/.jupyter/runtime /workspace/.local/bin /workspace/.cache \u0026#39; On the host, a playbook running chmod 0755 on a mounted NFS path failed with PermissionError: [Errno 1] Operation not permitted. Permissions belong to the server.\nRunning as the user, with root kept inside the container The launcher now starts every interactive shell and Jupyter process with the host UID and GID:\ndocker exec -it --user \u0026#34;$(id -u):$(id -g)\u0026#34; \u0026#34;$name\u0026#34; bash -lc \u0026#39;cd /workspace \u0026amp;\u0026amp; exec bash\u0026#39; Docker accepts a numeric UID for --user; only a user name must exist in the container. A UID with no entry in the image\u0026rsquo;s /etc/passwd still runs, and that is where I have no name! comes from. Tools that look up the user, sudo among them, break too.\nSo a separate root step runs first to make the image aware of the user:\ndocker exec --user 0 \u0026#34;$name\u0026#34; bash -c \u0026#34; grep -q \u0026#39;:x:$GID:\u0026#39; /etc/group || echo \u0026#39;$USER:x:$GID:\u0026#39; \u0026gt;\u0026gt; /etc/group id $UID \u0026gt;/dev/null 2\u0026gt;\u0026amp;1 || echo \u0026#39;$USER:x:$UID:$GID::/workspace:/bin/bash\u0026#39; \u0026gt;\u0026gt; /etc/passwd grep -q \u0026#39;^$USER:\u0026#39; /etc/shadow || echo \u0026#39;$USER:*:19999:0:99999:7:::\u0026#39; \u0026gt;\u0026gt; /etc/shadow command -v sudo \u0026gt;/dev/null || (apt-get update \u0026amp;\u0026amp; apt-get install -y sudo) echo \u0026#39;$USER ALL=(ALL) NOPASSWD:ALL\u0026#39; \u0026gt; /etc/sudoers.d/lab \u0026amp;\u0026amp; chmod 0440 /etc/sudoers.d/lab \u0026#34; Each line in that block fixed a failure we hit:\nWe first used useradd. Some images do not have it, so the script now writes to /etc/passwd and /etc/group directly. Images built on the Jupyter Docker Stacks start as jovyan (UID 1000). Without --user 0, the setup ran as jovyan and could not write /etc/passwd. The result was I have no name! and a broken sudo. A user that exists in passwd but not in shadow makes sudo fail with account validation failure. A non-expiring shadow line with a locked password fixes it. This sudo is container root only: users can apt install, but on /workspace it is the machine account and cannot read anyone\u0026rsquo;s dataset. Access to the Docker daemon itself is a separate problem, covered in a separate post on Docker authorization.\nThe jovyan case fails in a different way. The host has no ticket cache for UID 1000, so rpc.gssd has nothing to present. The user\u0026rsquo;s own UID is the only one that works correctly.\nThe UID Mismatch Two ID mapping engines, two numbers AD identifies users by SID. Every Linux system has to turn a SID into a number, and ours used two different engines.\nOn the clients, SSSD. With the AD provider, ldap_id_mapping is true by default. The sssd-ad(5) man page says the provider maps \u0026ldquo;UID and GID values from the objectSID parameter in Active Directory\u0026rdquo;, and that \u0026ldquo;When ID-mapping is enabled, the uidNumber and gidNumber attributes are ignored.\u0026rdquo; The default range runs from 200,000 to 2,000,200,000 in slices of 200,000. Each domain lands in a slice chosen by a hash of its SID, so the resulting UIDs are large and look arbitrary.\nOn TrueNAS, winbind with the RID backend. TrueNAS joins AD through Samba. Its documentation says that by default it \u0026ldquo;uses an algorithmic method based on the RID component of the user or group SID\u0026rdquo;, and lists the default domain range as 100000001 to 200000000. Our TrueNAS used this default. The Samba man page gives the formula: ID = RID - BASE_RID + LOW_RANGE_ID.\nBoth engines are deterministic, but they start from different bases. For a user with RID 1105, the client computed 1914801105 and TrueNAS computed 100001106. Neither was wrong, but they were not the same number. We first saw the two numbers side by side when we compared id on a client with the owner of the user\u0026rsquo;s files on TrueNAS.\nWhere the mismatch hits, layer by layer The example UIDs are the values before the fix.\nLayer Where the identity comes from Example UID What the mismatch does here Client host SSSD ID mapping from SID 1914801105 Consistent on the client. id -u returns it, and the launcher passes it to the container. If it is ever sent to TrueNAS as a number, it points to the wrong account or to none. Container docker exec --user \u0026lt;host UID\u0026gt; 1914801105 Without a passwd entry you get I have no name! and sudo breaks. As root or jovyan, NFS requests do not carry the user\u0026rsquo;s identity. NFS request rpc.gssd finds /tmp/krb5cc_\u0026lt;UID\u0026gt;, and the principal goes on the wire alice@EXAMPLE.LOCAL The number is only a lookup key on the client. The mismatch does not matter here, provided the ticket cache is named after the UID the process actually runs as. TrueNAS file system Principal to UID through winbind idmap, checked against ZFS ownership 100001106 The dataset must be owned by this number. If ownership is wrong, the user is locked out of their own 0700 dataset. TrueNAS API filesystem.setperm Whatever the provisioning code sends 100001106 (correct) or 1914801105 (wrong) Our name-based call did not apply. The dataset stayed root:root 755, so the owner could not write and other users could read. Sending the client\u0026rsquo;s UID would have been just as wrong. NFSv4 name mapping (idmapd) Server sends EXAMPLE\\alice@example.local, and the client maps it through NSS (SSSD) client: 1914801105 If the client\u0026rsquo;s NFSv4 domain does not match, the name does not map and the file shows as nobody. If it matches, the name maps to the client\u0026rsquo;s own UID and the mismatch disappears from view. Symptom 1: nobody:nogroup The client logs were full of this:\nnfsidmap: libnfsidmap: Unable to determine the NFSv4 domain; Using \u0026#39;localdomain\u0026#39; as the NFSv4 domain ... nfsidmap: nss_getpwnam: name \u0026#39;EXAMPLE\\alice@example.local\u0026#39; does not map into domain \u0026#39;localdomain\u0026#39; nfsidmap: nss_name_to_gid: name \u0026#39;EXAMPLE\\domain users@example.local\u0026#39; does not map into domain \u0026#39;localdomain\u0026#39; NFSv4 sends owners as user@domain strings, not numbers (nfsidmap(5)), and the client accepts only names from its own NFSv4 domain. If idmapd.conf does not set one, the client tries a DNS _nfsv4idmapdomain record, then the DNS domain. Ours found neither, fell back to localdomain, and every owner became nobody.\nThe fix was one line on every client, plus clearing the cache:\n# /etc/idmapd.conf [General] Domain = example.local sudo nfsidmap -c nfsidmap -c clears the keyring of cached mappings. The man page warns that a stale key alone can make ownership show as nobody.\nAfter that fix, ownership looked correct, and the mismatch became invisible: ls -l showed alice, ls -ln showed 1914801105, and TrueNAS stored 100001106.\nSymptom 2: the right owner, and still Permission denied At first, the share squashed all access to a single user (mapall), which hid ownership problems. Once we removed mapall and made the share Kerberos-only, isolation depended on one thing: each dataset\u0026rsquo;s owner had to be exactly the UID that TrueNAS maps the user\u0026rsquo;s principal to.\nThe first test after that change looked like this:\n$ stat -c \u0026#39;%U %u\u0026#39; ~/nfs-home alice 1914801105 $ touch ~/nfs-home/_t touch: cannot touch \u0026#39;/home/alice/nfs-home/_t\u0026#39;: Permission denied The client resolved the owner to the right user, because idmapd translated the name. The server still refused the write. Three things were wrong at the same time, and each one alone was enough to deny access:\nThe numbers differed. TrueNAS stored the files under its own UID, which the clients never used. The client did not present the user\u0026rsquo;s ticket. The cache name had a random suffix (/tmp/krb5cc_1914801105_XXXXXX), and rpc.gssd did not find it. The server mapped the principal to nobody. Even with the first two fixed, the user still got ls: cannot open directory ...: Permission denied. As a diagnostic step, we set the directory to 777 for a moment, and it opened. The user was getting in through the \u0026ldquo;other\u0026rdquo; bits, not as the owner. On TrueNAS, getent passwd 'EXAMPLE\\alice' resolved, but the bare getent passwd alice returned nothing. The fix section covers all three.\nSymptom 3: a new dataset owned by root:root 755 The provisioning code set permissions by name:\npayload = { \u0026#34;path\u0026#34;: mount_path, \u0026#34;uid\u0026#34;: None, \u0026#34;user\u0026#34;: \u0026#34;EXAMPLE\\\\alice\u0026#34;, \u0026#34;group\u0026#34;: \u0026#34;EXAMPLE\\\\domain users\u0026#34;, \u0026#34;mode\u0026#34;: \u0026#34;700\u0026#34;, \u0026#34;options\u0026#34;: {\u0026#34;recursive\u0026#34;: True, \u0026#34;traverse\u0026#34;: False}, } On our TrueNAS version the call was rejected, and new datasets stayed root:root 755. Nobody noticed, because the provisioning code caught the error, logged it as non-critical, and carried on. Older datasets looked fine only because they had been fixed by hand. Under a Kerberos-only share, a root:root 755 dataset means the owner cannot write to their own home and any other authenticated user can read it. Newer TrueNAS API references list user and group name fields for filesystem.setperm, so name-based calls may behave differently on your version. Numeric IDs avoid the question entirely.\nWhy this stays quiet No error ever mentioned a UID. Every tool showed the right name, the symptoms had different names in different layers (nobody, Permission denied on a file you own, root:root 755, I have no name!), and Kerberos kept the number off the wire. You only see the mismatch when you put the two numbers side by side.\nThe Fix The fix had three parts. First, make every layer carry the right identity. Second, make the two numbers equal. Third, fix the two lookups that still turned the user into nobody.\nPart 1: the right identity at every hop Names on the wire. Every client gets Domain = example.local in idmapd.conf and runs nfsidmap -c, so the owner strings from TrueNAS resolve through SSSD to local users. Principals for access. The share accepts only KRB5, with no mapall/maproot. The clients mount with sec=krb5, run rpc.gssd, keep the ticket cache at a predictable /tmp/krb5cc_%U, and renew it. Each of these broke at least once after the share became Kerberos-only: The pam_mount options did not include sec=krb5, so the mount was refused. rpc-gssd was not running on every client. It has to be enabled. rpc.gssd logged ERROR: unable to resolve 192.0.2.10 to hostname: Name or service not known. Without a reverse lookup it cannot build the NFS service principal for the server, and the mount fails. A PTR record in DNS fixes it, or an /etc/hosts entry with the server\u0026rsquo;s FQDN. Some users got ls: cannot access '/home/alice/nfs-home/': Key has expired. They had logged in while SSSD could not reach the KDC, so they had no valid ticket, and nothing renewed it. The renewal settings shown earlier fix this, and the user has to log out and back in once. The user\u0026rsquo;s UID inside the container. Every process that touches /workspace runs with --user \u0026lt;host UID\u0026gt;:\u0026lt;host GID\u0026gt;. Root work happens in a separate --user 0 exec that touches only the container\u0026rsquo;s own files. Ownership from the server\u0026rsquo;s own lookup. Provisioning asks TrueNAS which UID it uses for the user, then sets ownership numerically: uid_resp = self._post(\u0026#34;/user/get_user_obj\u0026#34;, {\u0026#34;username\u0026#34;: username}) user_obj = uid_resp.json() or {} uid, gid = user_obj.get(\u0026#34;pw_uid\u0026#34;), user_obj.get(\u0026#34;pw_gid\u0026#34;) if not isinstance(uid, int) or not isinstance(gid, int): return {\u0026#34;success\u0026#34;: False, \u0026#34;message\u0026#34;: \u0026#34;could not resolve uid/gid\u0026#34;} payload = { \u0026#34;path\u0026#34;: mount_path, \u0026#34;mode\u0026#34;: \u0026#34;700\u0026#34;, \u0026#34;uid\u0026#34;: uid, \u0026#34;gid\u0026#34;: gid, \u0026#34;options\u0026#34;: {\u0026#34;recursive\u0026#34;: True, \u0026#34;traverse\u0026#34;: False}, } resp = self._post(\u0026#34;/filesystem/setperm\u0026#34;, payload) user.get_user_obj returns the struct passwd fields for a user, including pw_uid and pw_gid, and it works for Active Directory accounts too. The TrueNAS API documents uid and gid in filesystem.setperm as numeric owner IDs. If TrueNAS cannot resolve the user, provisioning stops instead of leaving a dataset owned by root.\nPart 2: align the UIDs by moving the TrueNAS range Both engines add the user\u0026rsquo;s RID to a fixed base. TrueNAS uses range_low - BASE_RID. SSSD uses the start of the slice the domain landed in. The sssd-ad(5) page requires ldap_idmap_range_size to be at least the highest RID in the domain, so every user fits in that one slice. If TrueNAS starts its range at the same base, the numbers become equal.\nWe found the SSSD base by subtracting the RID from the client\u0026rsquo;s UID, for a user and for two groups. All three gave the same number, which confirmed that the client\u0026rsquo;s formula was base + RID:\nalice 1914801105 - 1105 = 1914800000 domain users 1914800513 - 513 = 1914800000 another group 1914801158 - 1158 = 1914800000 Then we changed the TrueNAS side:\nIn the Active Directory idmap settings for the domain, keep the RID backend and set Range Low to the base and Range High to the base plus the SSSD slice size (200,000). The RID backend\u0026rsquo;s SSSD compatibility option stays off; the range is already aligned. TrueNAS would not save idmap changes while the directory service was enabled. Disable the directory service, save, then enable it again. Enabling it makes TrueNAS rejoin the domain. The account used for the join needs write access to the TrueNAS computer object. Without it, the join fails with Insufficient access during LDB_MODIFY. A KRB5KDC_ERR_PREAUTH_FAILED at this step means a wrong password, a wrong UPN format, or a clock difference larger than Kerberos allows. Verify: wbinfo -i 'EXAMPLE\\alice' on TrueNAS must show the same UID as id -u alice on a client. Re-own the existing data, because every AD UID on TrueNAS has changed. With the example numbers:\nclient (SSSD): alice RID 1105 -\u0026gt; 1914801105 TrueNAS before: alice RID 1105 -\u0026gt; 100001106 (range 100000001-200000000) TrueNAS after: alice RID 1105 -\u0026gt; 1914801105 (range 1914800000-1915000000) The re-own step is not optional. ZFS stores numeric owners, so every existing dataset still pointed to the old TrueNAS UIDs. Because the old and the new TrueNAS formulas are both base + RID, the shift is the same for every account: 1914800000 - 100000001 = 1814799999. We took a recursive ZFS snapshot first, then added that shift to every UID and GID in the old range:\nzfs snapshot -r pool/users@before-idmap-fix OLD_LOW=100000001; OLD_HIGH=200000000; DELTA=1814799999 for U in $(find /mnt/pool/users -printf \u0026#39;%U\\n\u0026#39; | sort -un); do [ \u0026#34;$U\u0026#34; -ge \u0026#34;$OLD_LOW\u0026#34; ] \u0026amp;\u0026amp; [ \u0026#34;$U\u0026#34; -le \u0026#34;$OLD_HIGH\u0026#34; ] \u0026amp;\u0026amp; find /mnt/pool/users -uid \u0026#34;$U\u0026#34; -exec chown -h \u0026#34;$((U + DELTA))\u0026#34; {} + done # the same loop with %G, -gid and chgrp for groups Two details matter here. Each user\u0026rsquo;s home was its own child dataset, so find must not use -xdev, or it stops at the first dataset boundary. Local accounts with small UIDs are outside the old range, so the loop leaves them alone.\nAfter the change, the UID was the same for every user we checked on the clients, in the containers, and on TrueNAS. ls -ln now shows the same number on both sides. Even when a number does cross the boundary, it identifies the right person.\nPart 3: the two lookups that still returned nobody Aligned numbers were not enough. The user from Symptom 2 still got Permission denied, for two more reasons.\nOn the client, a ticket cache that rpc.gssd could find. SSSD had written the ticket to /tmp/krb5cc_1914801105_XXXXXX, with a random suffix. rpc.gssd did not use it. Setting krb5_ccname_template = FILE:%d/krb5cc_%U (shown earlier) gave the cache a fixed name. We rolled it out to every client and restarted SSSD. klist then showed /tmp/krb5cc_1914801105.\nOn TrueNAS, \u0026ldquo;Use Default Domain\u0026rdquo;. With this setting off, TrueNAS resolved only the domain-qualified name (EXAMPLE\\alice), and the bare alice returned nothing. Incoming principals ended up as nobody, which is why the 777 test opened the directory. With the setting on, the bare name resolved to the aligned UID and the 0700 home opened for its owner.\nChanging it had its own catch. TrueNAS refused to disable the directory service while it held a Kerberos user credential: Kerberos user credentials may not be stored for disabled directory services. We first switched the credential to the TrueNAS machine account\u0026rsquo;s Kerberos principal (its keytab), then disabled, changed the setting, and enabled again. That also removed the password and PREAUTH_FAILED problems from later rejoins.\nOld mappings are cached on both sides, so after the change we flushed them:\n# TrueNAS net cache flush nfsidmap -c midclt call service.restart nfs # clients: remount, or log out and back in A few limits apply to this trick:\nIt depends on SSSD\u0026rsquo;s slice choice. All clients must use the same SSSD ID mapping settings so they all land in the same slice. The man page warns that hash collisions can make slices differ between machines, so check more than one client. Users whose RID is above the SSSD range size do not fit the scheme. The man page says lookups fail for them. The TrueNAS docs warn that the domain range \u0026ldquo;must not overlap with local accounts or trusted domain ranges\u0026rdquo;, and that idmap mistakes can assign permissions to the wrong users. Make the change in a maintenance window and verify before users log in. How to Check Your Own Setup On the client, find out who you are:\nid # UID and GID as SSSD computed them getent passwd alice # the same, from NSS klist # the principal and the ticket cache path; it should end in krb5cc_\u0026lt;your UID\u0026gt; If the cache name does not contain your id -u, rpc.gssd cannot find it; check krb5_ccname_template.\nOn TrueNAS, find out who the server thinks you are (from the TrueNAS shell):\nid \u0026#39;EXAMPLE\\alice\u0026#39; midclt call user.get_user_obj \u0026#39;{\u0026#34;username\u0026#34;: \u0026#34;alice\u0026#34;}\u0026#39; Compare pw_uid with the client\u0026rsquo;s id -u. If they differ, subtract the RID from the client\u0026rsquo;s UID for two or three accounts. If the result is the same for all of them, that number is the client\u0026rsquo;s base, and the range change above will work.\nCheck the NFS mount and the name mapping:\nfindmnt -t nfs4 -o TARGET,SOURCE,OPTIONS # look for sec=krb5 and vers=4.x nfsidmap -d # effective NFSv4 domain; must not be \u0026#34;localdomain\u0026#34; grep -E \u0026#39;^\\s*Domain\u0026#39; /etc/idmapd.conf systemctl is-active rpc-gssd journalctl -t nfsidmap --since today # \u0026#34;does not map into domain\u0026#34; means wrong domain If the share is Kerberos-only and you still get Permission denied on a directory you own, check the server\u0026rsquo;s view of the bare user name (on TrueNAS, getent passwd alice without the domain), and check that klist shows a cache without a random suffix. To get more detail, stop rpc-gssd and run rpc.gssd -f -vvv in the foreground while you repeat the access.\nCheck ownership both ways:\nls -l ~/nfs-home # names: should show alice, never nobody ls -ln ~/nfs-home # numbers: after alignment, the same UID as on TrueNAS On TrueNAS, ls -ln /mnt/pool/users/alice should show the same UID with mode drwx------.\nInside the container:\ndocker exec --user \u0026#34;$(id -u):$(id -g)\u0026#34; mybox sh -c \u0026#39;id; whoami; ls -ln /workspace | head\u0026#39; whoami should print a name, and /workspace should show the same UID. Repeat with --user 0: listing /workspace should fail. If it succeeds, check the share\u0026rsquo;s maproot and the dataset\u0026rsquo;s mode.\nAlternatives SSSD on POSIX attributes. Store uidNumber/gidNumber in AD for every user, set ldap_id_mapping = False in SSSD, and select the AD (RFC2307) idmap backend on TrueNAS. Both sides read the same stored number, independent of any slice or range. The costs:\nEvery account needs those attributes, assigned uniquely and forever by your provisioning. The SSSD man page recommends replicating them to the Global Catalog for performance. The man page warns that changing ID mapping options \u0026ldquo;will cause user and group IDs to change\u0026rdquo;, so every home directory on the clients has to be re-owned. Two UID spaces. Leave the numbers different and rely on Kerberos plus idmapd names, always setting ownership from the server\u0026rsquo;s own lookup. Nothing is migrated, but everyone must know never to send a client UID to the server, which rules out sec=sys, NFSv3, and careless numeric backups. We ran like this for a while; it is how the mismatch stayed hidden. (Replacing SSSD with winbind on the clients would also give one algorithm on both sides, at the cost of a new identity stack.)\nThe approach we used: move the TrueNAS RID range. It changes one setting on one server, and it touches neither AD nor the clients. The cost is a one-time re-own of the existing datasets, a rejoin of the NAS to the domain, and a dependency on SSSD\u0026rsquo;s slice. If the SSSD ID mapping settings change, or a client lands in a different slice, the mismatch comes back. For an environment with one NAS and identical clients, that was the cheapest way to get one UID per user everywhere.\nConclusion With NFSv4 and Kerberos, identity on the wire is a principal plus a name string, and a numeric UID exists only inside each system. That let our clients and our NAS disagree about every UID with nothing visibly wrong. The problems appeared where a number or a name crossed a boundary: an idmapd domain that defaulted to localdomain, a ticket cache that rpc.gssd could not find, a server that could not resolve the bare user name, a permission call that failed silently, and containers that ran as root or as the image\u0026rsquo;s built-in user instead of the person at the keyboard.\nThree rules came out of this:\nRun container processes that touch Kerberos NFS as the user\u0026rsquo;s host UID, and keep root work inside the container. Set ownership on the server with the UID the server itself resolves. Compare id -u on the client with the server\u0026rsquo;s UID for the same user. If they differ, compute the client\u0026rsquo;s base (UID minus RID), start the server\u0026rsquo;s RID range at that base, and re-own the data. References rpc.gssd(8), credential cache lookup and machine credentials for UID 0: https://man7.org/linux/man-pages/man8/rpc.gssd.8.html nfsidmap(5), NFSv4 user@domain names, -c, -d, stale keys showing as nobody: https://man7.org/linux/man-pages/man5/nfsidmap.5.html idmapd.conf(5), Domain and its default: https://linux.die.net/man/5/idmapd.conf sssd-ad(5), ldap_id_mapping, ID ranges and slices, ldap_idmap_range_size: https://man.archlinux.org/man/sssd-ad.5.en sssd-krb5(5), krb5_ccname_template, renewal options: https://man.archlinux.org/man/sssd-krb5.5.en Samba idmap_rid(8): https://www.samba.org/samba/docs/current/man-html/idmap_rid.8.html TrueNAS, Configuring Active Directory (idmap backends and warnings): https://www.truenas.com/docs/scale/credentials/directoryservices/configad/ TrueNAS, Directory Services Screens (Range Low/High, default ranges): https://www.truenas.com/docs/scale/credentials/directoryservices/directoryservicesscreens/ TrueNAS API, filesystem.setperm: https://api.truenas.com/v25.10/api_methods_filesystem.setperm.html TrueNAS API, user.get_user_obj: https://api.truenas.com/v25.10/api_methods_user.get_user_obj.html Docker, running containers (default user, --user, numeric UIDs): https://docs.docker.com/engine/containers/run/ Docker CLI, docker container exec: https://docs.docker.com/reference/cli/docker/container/exec/ ","permalink":"https://onurbaydas.online/posts/nfs-uid-mismatch-en/","summary":"\u003ch2 id=\"introduction\"\u003eIntroduction\u003c/h2\u003e\n\u003cp\u003eOn a set of shared, multi-user Linux workstations, several things broke in the same few weeks. Files on a user\u0026rsquo;s NFS home showed up as \u003ccode\u003enobody:nogroup\u003c/code\u003e. Once we tightened the share, a user could no longer write to their own home, even though \u003ccode\u003els\u003c/code\u003e showed them as the owner. A freshly provisioned home dataset came out owned by \u003ccode\u003eroot:root\u003c/code\u003e with mode \u003ccode\u003e755\u003c/code\u003e instead of belonging to the user with mode \u003ccode\u003e700\u003c/code\u003e. A user\u0026rsquo;s Jupyter container opened with the prompt \u003ccode\u003eI have no name!\u003c/code\u003e and \u003ccode\u003esudo\u003c/code\u003e refused to work.\u003c/p\u003e","title":"One User, Two UIDs: NFSv4, Kerberos, and Docker on a Domain-Joined TrueNAS"},{"content":"Bu yazının Türkçesi: OPA ile Docker Authorization\n1. Introduction I set up opa-docker-authz to restrict Docker on shared, multi-user Linux workstations. After a routine policy update, I noticed that --privileged containers were silently running. This article summarizes that incident and what I found in the source code afterward.\nDocker authorization plugins let you evaluate every API request sent to the Docker daemon against a policy before the daemon processes it. opa-docker-authz is a plugin that implements this mechanism with OPA (Open Policy Agent) and its policy language, Rego.\nThis article covers:\nHow the Docker authorization plugin mechanism works, and how to install opa-docker-authz. What an example policy guarantees and what it doesn\u0026rsquo;t. Four pitfalls: silent fail-open, policy updates, Rego syntax and version compatibility, and fields the policy can\u0026rsquo;t see. Protections an authorization plugin can\u0026rsquo;t provide by design. A production checklist. Fail-open means a control mechanism lets a request through on error instead of rejecting it. The main subject of this article is that this can happen without a single error or warning.\n2. Scenario and Goal The scenario: multi-user, shared Linux workstations. Users need Docker, but they shouldn\u0026rsquo;t be able to access each other\u0026rsquo;s data or the host.\nThe core tension is this: the Docker daemon (dockerd) runs as root. The daemon is the background service that handles Docker API requests. Anyone who can reach the daemon\u0026rsquo;s socket can ask it to do work with root privileges. Docker\u0026rsquo;s official documentation also states that the docker group grants a user root-level privileges. In this scenario, users have access to the socket.\nThe goal is a single rule:\nEach user can bind only their own workspace (/home/\u0026lt;user\u0026gt;/workspace) to /workspace inside a container. No other bind mounts, no privileged containers, no host namespaces. GPU access is allowed.\nA bind mount attaches a directory on the host directly into a container. The container sees that directory as part of its own file system. If the container runs as root, it can read and write everything in that directory.\nExpected usage looks roughly like this:\ndocker run -d \\ --gpus all \\ -v \u0026#34;/home/\u0026lt;user\u0026gt;/workspace\u0026#34;:/workspace \\ \u0026#34;\u0026lt;image\u0026gt;\u0026#34; Because users have terminal access, the policy has to account for every request that can reach the Docker API, not just this command.\nThe approach in this article is to put an authorization layer in front of a standard dockerd running as root. Alternatives such as rootless Docker and userns-remap are covered in Section 10.\n3. How Docker Authorization Plugins Work Docker\u0026rsquo;s default access model is all-or-nothing. Any client that can reach the daemon can make any API call. Authorization plugins add a decision point to this model. Before processing each API request, the daemon asks the configured plugins whether they allow it.\nRequest flow:\nsequenceDiagram participant C as docker CLI / client participant D as dockerd (API server) participant M as Authz middleware participant P as opa-docker-authz plugin participant O as OPA (inside the plugin) participant H as API handler C-\u0026gt;\u0026gt;D: HTTP request (e.g. POST /v1.xx/containers/create) D-\u0026gt;\u0026gt;M: pass request to middleware alt chain is empty M-\u0026gt;\u0026gt;H: forward directly (no check) H--\u0026gt;\u0026gt;C: response else plugin in chain M-\u0026gt;\u0026gt;P: AuthZReq (Method, URI, headers, JSON body, User) P-\u0026gt;\u0026gt;P: read policy file from disk P-\u0026gt;\u0026gt;O: evaluate data.docker.authz.allow with input O--\u0026gt;\u0026gt;P: true / false / undefined / error P--\u0026gt;\u0026gt;M: Allow: true or false + message alt allowed M-\u0026gt;\u0026gt;H: process request H--\u0026gt;\u0026gt;C: response else denied M--\u0026gt;\u0026gt;C: \u0026#34;authorization denied by plugin ...\u0026#34; end end Details that matter for the rest of the article:\nIf the chain is empty, there is no check. WrapHandler in Moby\u0026rsquo;s pkg/authorization/middleware.go passes the request straight to the handler when the plugin list is empty. The middleware has no mechanism to warn that \u0026ldquo;a plugin should be here but isn\u0026rsquo;t.\u0026rdquo; This is the foundation of the pitfall in Section 6.\nThe middleware fails closed. According to Docker\u0026rsquo;s official documentation, if a plugin in the chain can\u0026rsquo;t be reached, the request is denied. Docker also recommends that plugin authors fail closed. So a \u0026ldquo;the plugin crashed and everything opened up\u0026rdquo; scenario isn\u0026rsquo;t expected at the middleware level. The risk is the plugin being removed from the chain without any warning.\nUser identity is empty in most setups. The request sent to the plugin has User and AuthenticationMethod fields. According to the Moby source code, these fields are populated only if the request arrived over TLS and the client presented a client certificate. The username comes from the certificate\u0026rsquo;s CN field. For requests over the Unix socket, which covers nearly all local docker commands, both fields are empty strings. The policy can\u0026rsquo;t answer the question \u0026ldquo;who made this request?\u0026rdquo; The consequences are covered in Section 10.\nThe plugin doesn\u0026rsquo;t see everything. According to the official documentation:\nThe body is sent to the plugin only when it can be forwarded as JSON. For hijacked connections such as exec and attach, the plugin sees only the initial HTTP request. For streamed responses such as logs and events, it sees only the request. The gRPC endpoint that BuildKit uses isn\u0026rsquo;t subject to authorization. A policy that tries to restrict image builds through authorization can\u0026rsquo;t see this channel. An endpoint is a path in the Docker API that performs a specific operation, for example /containers/create.\n4. Installation opa-docker-authz is an authorization plugin with OPA embedded that you can install as a Docker managed plugin. The examples use the openpolicyagent/opa-docker-authz-v2:0.9 tag. This version ships with OPA 0.60. Versioning is covered separately in Section 8.\nInstallation:\ndocker plugin install --alias opa-docker-authz \\ --grant-all-permissions \\ openpolicyagent/opa-docker-authz-v2:0.9 \\ opa-args=\u0026#34;-policy-file /opa/policies/authz.rego\u0026#34; The policy path needs care. The plugin\u0026rsquo;s configuration mounts the host\u0026rsquo;s /etc/docker directory read-only at /opa inside the plugin. A file at /etc/docker/policies/authz.rego on the host appears as /opa/policies/authz.rego from inside the plugin. opa-args must contain the path inside the plugin, not the host path. If the path is wrong, the plugin allows every request (Section 6).\nFile and directory permissions:\ninstall -d -m 0755 -o root -g root /etc/docker/policies install -m 0644 -o root -g root authz.rego /etc/docker/policies/authz.rego To add the plugin to the daemon\u0026rsquo;s authorization plugin chain, use daemon.json:\n{ \u0026#34;authorization-plugins\u0026#34;: [\u0026#34;opa-docker-authz\u0026#34;] } If a daemon.json already exists, merge this key with the other settings. The order is: install the plugin first, then update daemon.json, then make the daemon re-read its configuration. You can restart the service for this. systemctl reload docker also rebuilds the chain and doesn\u0026rsquo;t stop running containers (Section 6).\n5. The Policy, Line by Line The policy below uses OPA 0.60\u0026rsquo;s classic Rego syntax.\npackage docker.authz default allow = false allow { not is_container_create } allow { is_container_create; not deny } is_container_create { input.Method == \u0026#34;POST\u0026#34; endswith(split(input.Path, \u0026#34;?\u0026#34;)[0], \u0026#34;/containers/create\u0026#34;) } package docker.authz and the allow rule form the decision point the plugin queries by default: data.docker.authz.allow.\ndefault allow = false is a safe default. If no allow rule matches, the decision is deny.\nThe next two lines define the policy\u0026rsquo;s architecture:\nIf the request is not a container create request, allow it. If it is a container create request and no deny rule fired, allow it. is_container_create uses endswith to skip the API version prefix (such as /v1.47/) and split to strip query parameters (?name=...).\nHard prohibitions:\ndeny { input.Body.HostConfig.Privileged == true } deny { count(input.Body.HostConfig.CapAdd) \u0026gt; 0 } deny { input.Body.HostConfig.PidMode == \u0026#34;host\u0026#34; } deny { input.Body.HostConfig.NetworkMode == \u0026#34;host\u0026#34; } deny { input.Body.HostConfig.IpcMode == \u0026#34;host\u0026#34; } deny { input.Body.HostConfig.UTSMode == \u0026#34;host\u0026#34; } deny { startswith(input.Body.HostConfig.UsernsMode, \u0026#34;host\u0026#34;) } deny { count(input.Body.HostConfig.Devices) \u0026gt; 0 } deny { s := input.Body.HostConfig.SecurityOpt[_]; contains(s, \u0026#34;unconfined\u0026#34;) } deny { s := input.Body.HostConfig.SecurityOpt[_]; contains(s, \u0026#34;no-new-privileges:false\u0026#34;) } deny { s := input.Body.HostConfig.SecurityOpt[_]; contains(s, \u0026#34;no-new-privileges=false\u0026#34;) } These rules block --privileged, --cap-add, sharing the host\u0026rsquo;s PID/network/IPC/UTS namespaces, disabling the user namespace, passing devices through with --device, and setting protections such as seccomp/AppArmor to unconfined. Each one on its own can be enough to escape from a container to the host.\nThe bind mount rule:\ndeny { b := input.Body.HostConfig.Binds[_]; not bind_ok(b) } deny { m := input.Body.HostConfig.Mounts[_]; m.Type == \u0026#34;bind\u0026#34;; not mount_ok(m) } bind_ok(b) { parts := split(b, \u0026#34;:\u0026#34;) parts[1] == \u0026#34;/workspace\u0026#34; seg := split(parts[0], \u0026#34;/\u0026#34;) count(seg) == 4 seg[1] == \u0026#34;home\u0026#34; seg[3] == \u0026#34;workspace\u0026#34; } mount_ok(m) { m.Target == \u0026#34;/workspace\u0026#34; seg := split(m.Source, \u0026#34;/\u0026#34;) count(seg) == 4 seg[1] == \u0026#34;home\u0026#34; seg[3] == \u0026#34;workspace\u0026#34; } In Docker, a bind mount can come from two separate fields. The -v source:target syntax writes to the HostConfig.Binds array, and the --mount type=bind,... syntax writes to the HostConfig.Mounts array. The policy checks both.\nWhen the source path is split on /, it must have exactly four parts: \u0026quot;\u0026quot;, \u0026quot;home\u0026quot;, the username, and \u0026quot;workspace\u0026quot;. The target must be /workspace. This also rejects deeper paths such as /home/\u0026lt;user\u0026gt;/workspace/subdir. The reason: Docker resolves symbolic links (symlinks) in the bind source. Before this rule was added, the following was seen in production: when a symlink pointing to the host\u0026rsquo;s root was placed inside the workspace and that subpath was bind-mounted, the host\u0026rsquo;s files could be read. The four-part rule closes off symlinks that come in through a subpath. The workspace directory itself being a symlink is a separate risk (Section 10).\nWhat it guarantees (verified on a live daemon): --privileged, --cap-add, binding the host root or /etc, the subpath symlink, binding the Docker socket into a container, --device, --ipc=host, and binding outside the workspace with --mount type=bind are all denied. Expected usage (the workspace bind and --gpus all) works. When the machine reboots, the plugin comes back and the rules keep being enforced.\nWhat it doesn\u0026rsquo;t guarantee is covered in Sections 9 and 10. Some of it comes from gaps in the policy, and some comes from the mechanism itself. First, though, consider the case where protection can shut off entirely even when the policy is correct.\n6. Pitfall 1: Silent Fail-Open Symptom The plugin shows as enabled in docker plugin ls, authorization-plugins is set in daemon.json, and the policy file is correct. Yet docker run --rm --privileged alpine true succeeds. Host bind mounts aren\u0026rsquo;t denied either. The daemon log has no error or warning.\nThis happens after running this pair of commands:\ndocker plugin disable --force opa-docker-authz docker plugin enable opa-docker-authz This pair is a common habit, used to \u0026ldquo;reload the plugin so it picks up the new policy.\u0026rdquo;\nMechanism: The Plugin Is Dropped from the Chain Three pieces of the Moby source code explain this behavior:\nThe Disable() function in daemon/pkg/plugin/backend_linux.go calls AuthzMiddleware.RemovePlugin(name) when a plugin of the authorization type is disabled. The plugin is removed from dockerd\u0026rsquo;s authorization plugin chain. The Enable() function in the same file has no call that adds the plugin back to the middleware. The plugin is enabled and docker plugin ls shows it as enabled, but the plugin isn\u0026rsquo;t in the chain. The config reload path in daemon/command/daemon.go calls authzMiddleware.SetPlugins(cfg.AuthorizationPlugins). This is where the chain is rebuilt from the list in daemon.json. systemctl reload docker sends SIGHUP to the daemon and triggers this path. With the chain empty, the middleware passes every request straight to the handler (Section 3). As a result, authorization is silently off after disable/enable. The daemon reports no error, the plugin looks enabled, and the policy file is correct, but no request reaches the plugin. The behavior is the same in the current Moby source.\nIn short: docker plugin disable turns authorization off, docker plugin enable doesn\u0026rsquo;t turn it back on, and systemctl reload docker does.\nHistory: Why Does Removal from the Chain Exist? Removal from the chain isn\u0026rsquo;t a bug; it\u0026rsquo;s a fix. Moby issue #31836, opened in 2017, reported that after an authorization plugin was disabled, the daemon still sent requests to that plugin. Because the plugin no longer responded, all API requests were blocked.\nThe fixing commit, 38de272bd4, added code to Disable() that removes an authorization-type plugin from the middleware chain. The same commit didn\u0026rsquo;t add code to Enable() that puts the plugin back in the chain. This first fix removed the plugin from the chain with different code; later the same year, moby #33655 (\u0026ldquo;Eliminate authz plugin disable race\u0026rdquo;) moved that code into today\u0026rsquo;s RemovePlugin function, and the behavior stayed the same. The silent fail-open described in this section is a side effect of that asymmetry in the fix.\nObserved Behavior Test results on a live daemon match the mechanism in the source code:\nStep --privileged test Policy written + systemctl reload docker Denied Policy changed + disable / enable Succeeded (fail-open) Directory set to 755, file to 644 + disable / enable Succeeded (fail-open) Plugin reinstalled (disable → rm → install) + systemctl reload docker Denied The behavior is the same in a current version: with Docker Engine 29.9.0 and opa-docker-authz v0.9, --privileged succeeds after disable/enable and is denied after systemctl reload docker.\nIn the last step, the chain is rebuilt by the reload, not the reinstall. A reinstall also includes a disable/enable cycle and doesn\u0026rsquo;t add the plugin back to the chain on its own. The plugin is added back by SetPlugins, that is, during the reload.\nThe daemon journal has a notable entry: the plugin allowed the POST .../plugins/opa-docker-authz/disable?force=1 request that disabled it (result: true). Because this policy only checks container creation, the disable request also passes through authorization and gets allowed. After this entry, there are no decision entries for the test requests: requests no longer reach the plugin.\nA Common Misdiagnosis: File Permissions A frequent explanation for this symptom goes like this: \u0026ldquo;If the policy file isn\u0026rsquo;t 644, the plugin can\u0026rsquo;t read it, runs with an empty policy, and allows everything.\u0026rdquo; This explanation is wrong for two reasons:\nIn the table above, protection was still off after disable/enable even once permissions were set to 644. The step that fixed it was the reload. opa-docker-authz reads the policy file with os.ReadFile. If the read fails (including a permission error), it denies the request. A permission problem produces a fail-closed result. A likely source of the confusion: the official openpolicyagent/opa CLI image runs as a non-root user. Running opa check with this image on a policy file with 600 permissions returns \u0026ldquo;permission denied.\u0026rdquo; That says nothing about the plugin\u0026rsquo;s behavior. The 755/644 permissions are a recommended hygiene rule so that separately run tools such as opa check can read the file, but they aren\u0026rsquo;t the cause of the fail-open.\nWhen the Plugin Explicitly Fails Open The plugin\u0026rsquo;s source code does have a branch that genuinely fails open. Before reading the policy file, the plugin checks it with os.Stat. If the file doesn\u0026rsquo;t exist, the plugin logs this message and allows the request:\nOPA policy file %s does not exist, failing open and allowing request So a typo in the path inside opa-args, or getting the /etc/docker → /opa translation wrong, shuts protection off completely. In this case, the plugin log does leave a trace.\nThe decision table derived from the source code:\nCondition Plugin decision Policy file missing Allow (fail-open), logged File unreadable (e.g., permission error) Deny Rego compile or evaluation error Deny allow undefined Deny Plugin not in the chain Plugin is never asked, allow Detection The plugin appearing \u0026ldquo;enabled\u0026rdquo; or the file hash being correct won\u0026rsquo;t catch this problem. A drift check that collects the policy file\u0026rsquo;s hash, the file mode, and the enabled state from docker plugin ls returns a \u0026ldquo;correct\u0026rdquo; result even during a fail-open. Don\u0026rsquo;t rely on configuration output such as docker info for the actual state of the chain either.\nThe reliable check is a behavioral test. With a regular user account:\ndocker run --rm --privileged alpine true This command should be denied with this message:\ndocker: Error response from daemon: authorization denied by plugin opa-docker-authz:latest: request rejected by administrative policy It isn\u0026rsquo;t enough for the command to merely fail. It can also fail because the image couldn\u0026rsquo;t be pulled or the daemon couldn\u0026rsquo;t be reached. Check that the output contains authorization denied. A version of this test that runs on a schedule is in Section 11.\n7. Pitfall 2: Updating the Policy According to its source code, the plugin reads the policy file from disk on every request. There is no cache. As soon as the file changes, the next request is evaluated against the new policy. Updating the policy doesn\u0026rsquo;t require reloading the plugin.\nThe real risk is the disable/enable pair making its way into update automation. Automation that disables and re-enables the plugin when the policy changes, and doesn\u0026rsquo;t reload because daemon.json didn\u0026rsquo;t change, turns authorization off on every machine it touches, each time it runs, until the next reload or restart. Even if the correct procedure is written down in the documentation, this risk goes unnoticed when the automation does something different.\nA safe policy update:\nCompile the new policy with the OPA version the plugin uses (opa check) and evaluate it against test inputs (opa eval). Put the file in place. Because the plugin reads the file on every request, if it sees a half-written file mid-write, that request is denied due to a compile error. To avoid this, write the new file under a temporary name in the same directory and swap it in with mv. Don\u0026rsquo;t touch the plugin. Run the --privileged test. If the plugin itself changed (version upgrade, reinstall), run systemctl reload docker once you\u0026rsquo;re done and repeat the behavioral test. If the policy forbids writes to the plugin API (Section 9), the plugin must first be removed from the chain via daemon.json before these operations.\nA pattern for rolling out authorization to multiple machines: back up → reload → roll back automatically if docker ps fails → --privileged test. It doesn\u0026rsquo;t stop running containers and can run on machines in parallel.\n#!/usr/bin/env bash set -u cfg=/etc/docker/daemon.json bak=\u0026#34;$cfg.bak.$(date +%s)\u0026#34; cp -a \u0026#34;$cfg\u0026#34; \u0026#34;$bak\u0026#34; # Merge authorization-plugins into the existing settings jq \u0026#39;. + {\u0026#34;authorization-plugins\u0026#34;: [\u0026#34;opa-docker-authz\u0026#34;]}\u0026#39; \u0026#34;$bak\u0026#34; \u0026gt; \u0026#34;$cfg.new\u0026#34; \\ \u0026amp;\u0026amp; mv \u0026#34;$cfg.new\u0026#34; \u0026#34;$cfg\u0026#34; systemctl reload docker # Roll back automatically if the daemon is unhealthy if ! docker ps \u0026gt;/dev/null 2\u0026gt;\u0026amp;1; then cp -a \u0026#34;$bak\u0026#34; \u0026#34;$cfg\u0026#34; systemctl reload docker echo \u0026#34;ROLLBACK\u0026#34; exit 1 fi echo -n \u0026#34;ps=OK \u0026#34; # Behavioral test if docker run --rm --privileged alpine true 2\u0026gt;\u0026amp;1 | grep -q \u0026#34;authorization denied\u0026#34;; then echo \u0026#34;priv=DENIED\u0026#34; else echo \u0026#34;priv=OPEN\u0026#34; exit 2 fi Don\u0026rsquo;t consider the rollout done until every machine reports ps=OK priv=DENIED.\n8. Pitfall 3: Rego Syntax and Plugin Version Version Status Rego syntax changed with OPA 1.0. if before rule bodies and contains for multi-value rules are now mandatory. The policy in Section 5 is written in classic (0.x) syntax.\nopa-docker-authz versions:\nPlugin OPA Managed plugin on Docker Hub v0.9 0.60.0 Yes (latest tag) v0.10 1.3.0 No v0.11 1.21.1 No The project is maintained, and new releases are published on GitHub. However, the newest tag you can pull from Docker Hub with docker plugin install is 0.9. Because 0.10 and later aren\u0026rsquo;t published on Docker Hub as managed plugins, using these versions requires building the plugin from source.\nThe result: a classic policy written for 0.9 won\u0026rsquo;t compile when the plugin is upgraded to 0.10+. opa check with OPA 1.21.1:\nrego_parse_error: `if` keyword is required before rule body The plugin denies the request on a compile error. The result is fail-closed, but every request, including docker ps, is denied and Docker becomes unusable.\nBridge: import rego.v1 For a policy that works on both versions, use import rego.v1. This import was added in OPA 0.59.0. Once the policy is converted to OPA 1.x syntax and this line is added at the top, both 0.60 and 1.x compile it.\npackage docker.authz import rego.v1 default allow := false allow if { is_container_create not deny } deny if input.Body.HostConfig.Privileged == true authz-hardened-v1.rego is in this form and was evaluated with all test inputs on OPA 0.60.0 and 1.21.1. Both versions return the same decisions. Converting the policy to this form before upgrading the plugin is recommended.\nA Rego Pitfall: An Error Can Turn into \u0026ldquo;Allow\u0026rdquo; By default, OPA treats a runtime error in a built-in function (for example, an invalid regex) not as an error but as undefined. The plugin also calls OPA without the StrictBuiltinErrors option.\nIf a built-in errors inside a deny rule, the rule becomes undefined, meaning it doesn\u0026rsquo;t fire. not deny is then true. In the deny-list + not deny pattern, an error means allow.\nTest: with a deny rule containing a broken regex, a privileged input returns allow = true. Running the same evaluation with --strict-builtin-errors returns eval_builtin_error.\nMitigations:\nRun policy tests in CI (continuous integration) with opa eval --strict-builtin-errors. In deny rules, avoid built-ins that can error (regex, parsing functions) as much as possible. Where possible, use an allow-list instead of a deny-list. Describe what\u0026rsquo;s allowed, not what\u0026rsquo;s forbidden. That way, an undefined result means deny. 9. Pitfall 4: What the Policy Can\u0026rsquo;t See The policy in Section 5 and its hardened version were tested against 19 test scenarios, both with opa eval and on a live Docker daemon.\nTest input Original Hardened Own workspace allowed allowed Another user\u0026rsquo;s workspace allowed allowed --privileged denied denied /etc bind denied denied Workspace subpath denied denied Root directory via --mount type=bind denied denied Volume mount with driver options allowed denied --volumes-from allowed denied --gpus allowed allowed Bodyless create request allowed denied Plugin disable request allowed denied Plugin list (GET) allowed allowed volume create with driver options allowed denied Plain volume create allowed allowed Bodyless volume create allowed denied exec allowed allowed exec --privileged allowed denied Path containing .. allowed denied --cap-add denied denied Live daemon tests were run with plugin v0.9 (OPA 0.60). In the opa eval evaluation, the hardened column is the same for the classic-syntax authz-hardened.rego on OPA 0.60, and for authz-hardened-v1.rego on both OPA 0.60 and 1.21.1. The rows for another user\u0026rsquo;s workspace and exec are the subject of Section 10.\nThere\u0026rsquo;s an inconsistency in the plain volume create row. When a named volume is attached with -v name:/path, it lands in HostConfig.Binds, doesn\u0026rsquo;t match the /home/... format, hits the bind rule, and is denied. When the same volume is attached with --mount type=volume, it has no driver configuration, so it\u0026rsquo;s allowed. Plain volumes don\u0026rsquo;t point to a host directory, so this is harmless. A policy that also wants to allow -v should add a separate branch to bind_ok for named volumes.\nVolume-Type Mounts The original policy checks HostConfig.Mounts entries only if Type == \u0026quot;bind\u0026quot;. Docker\u0026rsquo;s local volume driver, however, supports creating a volume that points to a directory on the host through driver options. Such a mount arrives as Type == \u0026quot;volume\u0026quot; and isn\u0026rsquo;t covered by the bind rule. The container can access an arbitrary directory on the host.\nThe fix: deny volume mounts that contain a driver configuration.\ndeny { m := input.Body.HostConfig.Mounts[_]; m.Type == \u0026#34;volume\u0026#34;; m.VolumeOptions.DriverConfig } volume create Options You can reach the same result by first creating a volume with driver options through a separate POST /volumes/create request and then mounting it. Because the original policy allows every endpoint other than container create, this request isn\u0026rsquo;t checked.\nThe fix: make volume creation a separate decision point and accept only requests without driver options. Volumes without options remain allowed.\nallow { is_volume_create; volume_ok } is_volume_create { input.Method == \u0026#34;POST\u0026#34; endswith(split(input.Path, \u0026#34;?\u0026#34;)[0], \u0026#34;/volumes/create\u0026#34;) } volume_ok { is_object(input.Body); count(object.get(input.Body, \u0026#34;DriverOpts\u0026#34;, {})) == 0 } The rule is intentionally written as an allow-list. A deny-list rule such as volume_deny { count(input.Body.DriverOpts) \u0026gt; 0 } doesn\u0026rsquo;t fire when the body doesn\u0026rsquo;t arrive, and the request goes through (see Bodyless Requests below). The is_object check also denies the cases where the body doesn\u0026rsquo;t arrive at all or arrives as null. The Docker CLI always sends a body with volume create requests, and DriverOpts is an empty object when there are no options, so normal usage isn\u0026rsquo;t affected.\n--volumes-from HostConfig.VolumesFrom lets a container inherit all the mounts of another container. On a shared machine, this means inheriting another user\u0026rsquo;s workspace from a container that has it mounted. The original policy doesn\u0026rsquo;t look at this field.\ndeny { count(input.Body.HostConfig.VolumesFrom) \u0026gt; 0 } Path Normalization The policy only splits the path on /; it doesn\u0026rsquo;t resolve it. /home/../workspace splits into four parts and satisfies the rule. This example doesn\u0026rsquo;t directly give access to sensitive data, but it shows that the policy evaluates the path as text. The hardened version rejects an empty value, ., and .. in the username segment:\nbind_ok(b) { parts := split(b, \u0026#34;:\u0026#34;) parts[1] == \u0026#34;/workspace\u0026#34; seg := split(parts[0], \u0026#34;/\u0026#34;) count(seg) == 4 seg[1] == \u0026#34;home\u0026#34; not {\u0026#34;\u0026#34;, \u0026#34;.\u0026#34;, \u0026#34;..\u0026#34;}[seg[2]] seg[3] == \u0026#34;workspace\u0026#34; } The plugin\u0026rsquo;s input has a Resolved field that is expected to carry the path with symlinks resolved. In the managed plugin, however, this field arrives empty. The plugin does the resolution on its own file system, and the host\u0026rsquo;s /home directory doesn\u0026rsquo;t exist inside the plugin. For this reason, Resolved can\u0026rsquo;t be used as a symlink defense.\nBodyless Requests This is the most insidious blind spot of the deny-list pattern. All the deny rules look at fields under input.Body.HostConfig. If the body doesn\u0026rsquo;t reach the plugin, or the plugin can\u0026rsquo;t parse it, no deny rule fires, not deny is true, and the request goes through.\nDon\u0026rsquo;t assume the plugin will always parse the body. Docker\u0026rsquo;s documentation also states that the body isn\u0026rsquo;t forwarded to the plugin in every case. The fix is one line:\ndeny { not input.Body.HostConfig } This rule denies a container create request that has no HostConfig. A normal docker run always sends HostConfig, so the rule doesn\u0026rsquo;t affect legitimate use.\nThis blind spot isn\u0026rsquo;t theoretical. In CVE-2024-41110 (CVE: Common Vulnerabilities and Exposures), a specially crafted API request could make the daemon forward the request to the authorization plugin without its body. Plugins that make decisions based on the body could allow a request they would normally deny. The vulnerability was first fixed in January 2019, in Docker Engine v18.09.1. Because the fix wasn\u0026rsquo;t carried forward to later major versions, it came back as a regression. According to the advisory, the affected versions are v19.03.15, v20.10.27, v23.0.14, v24.0.9, v25.0.5, v26.0.2, v26.1.4, v27.0.3, v27.1.0, and earlier versions on each of those branches. The advisory gives patched versions for only three branches: versions after v23.0.14, v26.1.4, and v27.1.0. For docker-ce, the fix is in v27.1.1. The advisory gives no patched version for the 19.03, 20.10, 24.0, 25.0, 26.0, and 27.0 branches. If you\u0026rsquo;re on one of these branches, upgrade to a patched version.\nThe deny { not input.Body.HostConfig } rule catches exactly this case: in a bodyless create request, HostConfig is undefined and the request is denied. Even if the engine is patched, the rule should stay as defense in depth, because the policy\u0026rsquo;s security shouldn\u0026rsquo;t depend on a single fix in the engine.\nThe same blind spot exists in every deny-list rule that looks at the body. When you add a decision point other than container create to the policy (volume create, exec), test separately that it also denies bodyless requests.\nThe Plugin API In the journal entry from Section 6, the request to disable the plugin passes through authorization and is allowed. This means any user who can reach the socket can turn off authorization entirely. It\u0026rsquo;s a direct consequence of the mechanism in Section 6.\nThe fix: deny every operation on the plugin API other than reads.\nallow { not is_container_create; not is_volume_create; not is_plugin_write } is_plugin_write { input.Method != \u0026#34;GET\u0026#34;; input.PathArr[1] == \u0026#34;plugins\u0026#34; } is_plugin_write { input.Method != \u0026#34;GET\u0026#34;; startswith(input.PathArr[1], \u0026#34;v\u0026#34;); input.PathArr[2] == \u0026#34;plugins\u0026#34; } The second rule catches version-prefixed paths (/v1.47/plugins/...).\nThe operational consequence: this rule also prevents administrators from disabling the plugin over the socket. To upgrade or reinstall the plugin, first remove the authorization-plugins key from daemon.json and reload. When you\u0026rsquo;re done, put the key back and reload again. This trade-off also makes the dangerous disable/enable habit from Section 7 impossible.\nexec --privileged docker exec --privileged starts a privileged process in a running container. This request goes to the POST /containers/{id}/exec endpoint, and its body contains a Privileged: true field. Because the policy in Section 5 only checks container creation, this request is unrestricted. So privileged access that was denied at the create stage can be obtained at the exec stage.\nThis part doesn\u0026rsquo;t require identity, so policy can close it:\nallow { not is_container_create; not is_volume_create; not is_plugin_write; not is_exec_create } allow { is_exec_create; exec_ok } is_exec_create { input.Method == \u0026#34;POST\u0026#34; input.PathArr[count(input.PathArr) - 1] == \u0026#34;exec\u0026#34; } exec_ok { input.Body.Privileged == false } The first line replaces the general allow rule from The Plugin API.\nThe rule is intentionally written as an allow-list. The Docker CLI always sends the Privileged field in exec requests, so a normal docker exec keeps working. Because it\u0026rsquo;s an allow-list, a bodyless request or one missing the field is denied.\nA deny-list rule such as exec_deny { not input.Body } doesn\u0026rsquo;t do the job. When the plugin can\u0026rsquo;t parse the body, the input.Body field can arrive as null. null is a defined value in Rego, not false, so not input.Body is false. A bodyless privileged exec request passes such a rule. The create-side deny { not input.Body.HostConfig } rule isn\u0026rsquo;t affected by this problem: the HostConfig field of a null body is undefined, so the rule fires.\n--gpus and DeviceRequests --gpus writes to HostConfig.DeviceRequests, not HostConfig.Devices. The Devices rule doesn\u0026rsquo;t block GPU access. In this scenario, GPU access is part of the goal, so the field was intentionally left open. A policy that wants to restrict GPUs has to handle DeviceRequests separately.\nOther Unchecked Fields CgroupnsMode, CgroupParent, Sysctls, Runtime, and PortBindings aren\u0026rsquo;t checked by either policy. They were considered lower risk. By the nature of the deny-list approach, the list has to be reviewed whenever a new field is added to the Docker API.\n10. What an Authorization Plugin Can\u0026rsquo;t Provide Two rows in the hardened table are still open: another user\u0026rsquo;s workspace and exec. exec --privileged can be closed with policy (Section 9). But restrictions that depend on the question \u0026ldquo;whose container is this?\u0026rdquo; can\u0026rsquo;t be closed by improving the policy.\nNo Identity For requests over the Unix socket, the User field is empty (Section 3). The policy can check the format of the /home/\u0026lt;user\u0026gt;/workspace path, but it can\u0026rsquo;t know whether the requester is that \u0026lt;user\u0026gt;. A user can bind another user\u0026rsquo;s workspace into their own container.\nHome directories with 0700 permissions don\u0026rsquo;t stop this. The files are accessed by the daemon, which runs as root. For directories on local disk, this means direct read and write access. If workspaces are mounted over NFS (Network File System) with sec=krb5, whether the root daemon can access another user\u0026rsquo;s data depends on Kerberos identity mapping.\nThe same cause creates one more risk. Because users own their home directories, a user can delete the workspace directory and replace it with a symlink that points somewhere else on the host. The path format passes the policy, and because Docker resolves symlinks in the bind source (Section 5), whatever the symlink points to gets mounted. If the workspace is an NFS mount point, it can\u0026rsquo;t be deleted. When the mount is absent, or the workspace is a local directory, this is possible. A policy that looks at path format can\u0026rsquo;t close this risk.\nCarrying identity with TLS client certificates is theoretically possible. However, it requires users to connect over TLS with their own certificates instead of the local socket. For a shared workstation, that\u0026rsquo;s a separate infrastructure project.\nexec, commit, and rm Are Unrestricted The policy only checks container creation. And since there\u0026rsquo;s no identity, you can\u0026rsquo;t write a rule based on \u0026ldquo;whose container is this?\u0026rdquo; either. A user can:\nexec into another user\u0026rsquo;s container as root, turn that container into an image with commit and take it out, stop or delete it. You can ban exec entirely, but then users can\u0026rsquo;t get into their own containers either. Without ownership information, you can\u0026rsquo;t write a selective rule.\nThe docker Group Is Root-Equivalent According to Docker\u0026rsquo;s official documentation, the docker group grants a user root-level privileges, and only trusted users should be able to control the Docker daemon. An authorization plugin doesn\u0026rsquo;t change this; it only closes some paths. The list of those paths can easily end up incomplete. The plugin can silently drop out of service through an administrator mistake or, if the policy isn\u0026rsquo;t strict enough, through a regular user\u0026rsquo;s request.\nAuthorization Alone Is Not a Security Boundary It helps to split the risks into two categories:\nGaps by design: Access to the host through legitimate Docker features (such as --privileged, host bind mounts, and volume driver options). An authorization policy exists to close these. Runtime escape vulnerabilities: Escaping a container through security bugs in runc or the kernel. No policy closes these. The remedy is keeping runc, Docker, and the kernel up to date. An authorization plugin is a useful layer for the first category. But because it has no identity, can be dropped from the chain, and relies on a deny-list that can be incomplete, it isn\u0026rsquo;t enough on its own against untrusted users.\nAlternatives Rootless Docker: The daemon runs with the user\u0026rsquo;s own privileges. The root-equivalence problem largely goes away. Podman: A daemonless alternative that can run rootless. userns-remap: Maps root in the container to an unprivileged UID range on the host. A separate daemon per user: Each user gets their own socket and isolation. Kubernetes + admission control: Identity and authorization are built into the API server. The policy (for example, with OPA Gatekeeper) is written in a context that knows identity. A root-owned launcher: Instead of the Docker socket, users get a small root-owned tool that launches containers only with allowed parameters. Identity comes from the operating system. Every option has a cost. Rootless Docker can cause compatibility problems when added to an existing setup after the fact. Because userns-remap maps root in the container to a different UID on the host, it can break users\u0026rsquo; access to their NFS data under their own identities. GPU access can also be affected. For requirements that need identity, a root-owned launcher or a separate daemon per user is a sturdier foundation than an authorization plugin.\n11. Production Checklist Version\nRun Docker Engine 29.3.1 or later. This version patches CVE-2026-34040, which allows AuthZ plugins to be bypassed, and CVE-2026-33997, which allows privilege validation to be skipped during docker plugin install. If you have to stay on an older version, at least use one that is patched for CVE-2024-41110. The advisory gives patched versions only for the 23.0, 26.1, and 27.1 branches: versions after v23.0.14, v26.1.4, and v27.1.0 (docker-ce v27.1.1). No patched version is given for the 19.03, 20.10, 24.0, 25.0, 26.0, and 27.0 branches; if you\u0026rsquo;re on one of them, upgrade to a patched version (Section 9). Permissions\nSet the policy directory to 0755 root:root and the file to 0644 root:root. This isn\u0026rsquo;t the cause of the fail-open. It\u0026rsquo;s a recommended hygiene rule so that separately run tools such as opa check can read the file. Write the policy path as the path inside the plugin (/etc/docker/... → /opa/...). If the path is wrong, the plugin explicitly fails open. Behavioral Test\nAfter every installation, every policy change, and every plugin operation, run docker run --rm --privileged alpine true as a regular user. Check for the authorization denied message, not just the exit code. docker plugin ls, file hashes, and file modes don\u0026rsquo;t show the state of the chain. A drift check limited to these won\u0026rsquo;t catch a fail-open. Periodic Test and Alerting\nRun the behavioral test at regular intervals and raise an alert if it isn\u0026rsquo;t denied. This is the check that catches the situation in Section 6 the earliest: #!/usr/bin/env bash # Run as a non-root service account in the docker group. out=$(docker run --rm --privileged alpine true 2\u0026gt;\u0026amp;1) if ! grep -q \u0026#34;authorization denied\u0026#34; \u0026lt;\u0026lt;\u0026lt;\u0026#34;$out\u0026#34;; then logger -p auth.crit \u0026#34;docker-authz: privileged test NOT denied: ${out:-\u0026lt;succeeded\u0026gt;}\u0026#34; exit 1 fi Update Procedure\nFor policy changes, don\u0026rsquo;t touch the plugin. Change the file and test. Don\u0026rsquo;t use docker plugin disable/enable to update the policy. Before plugin operations (upgrade, reinstall), remove the plugin from daemon.json and reload. When you\u0026rsquo;re done, add it back, reload again, and repeat the behavioral test. Compare your automation scripts against the documented procedure. Deny write operations on the plugin API in the policy. Before upgrading the plugin version, convert the policy to the import rego.v1 form and run opa check with the target OPA version. Policy\nDeny bodyless requests: deny { not input.Body.HostConfig }. For decision points other than container create (volume create, exec), use an allow-list and test with a bodyless request. Check the volume type in Mounts, volume create options, and VolumesFrom. Reject ., .., and empty values in path segments. Test in CI with --strict-builtin-errors. Backup\nDon\u0026rsquo;t exclude the /var/lib/docker directory from backups, or explicitly add the plugin to your restore procedure. The plugin binary lives there. If daemon.json requires the plugin and the plugin is missing, validateAuthzPlugins fails and dockerd doesn\u0026rsquo;t start. The result is fail-closed: authorization doesn\u0026rsquo;t open up, but Docker stops entirely on that machine. Logging\nShip the plugin\u0026rsquo;s decision logs to a central location. An allow decision on the plugin API, followed by decision logs that stop, is the first sign of the plugin being dropped from the chain. Define a separate alert rule for the failing open and allowing request message. 12. Conclusion You know a security control works by testing whether it actually blocks, not because it looks \u0026ldquo;enabled.\u0026rdquo; The plugin can look enabled, and the policy file and permissions can be correct. Despite all that, the authorization plugin chain can be empty. A one-line docker run --privileged test reveals this in a second. docker plugin disable turns authorization off, enable doesn\u0026rsquo;t turn it back on, and systemctl reload docker does.\nA Docker authorization plugin doesn\u0026rsquo;t change the fact that the docker group is root-equivalent. It\u0026rsquo;s a layer that doesn\u0026rsquo;t know identity, checks only some endpoints, and can silently drop out of service. It\u0026rsquo;s useful for preventing accidents among trusted users. Separating untrusted users requires a sturdier foundation, such as rootless operation, per-user isolation, or a control plane that knows identity.\nSince that incident, I run the behavioral test periodically. You can\u0026rsquo;t know the difference between \u0026ldquo;looks enabled\u0026rdquo; and \u0026ldquo;works\u0026rdquo; without testing it.\n13. References Docker, Access authorization plugin: https://docs.docker.com/engine/extend/plugins_authorization/ Docker, Linux post-installation steps (docker group): https://docs.docker.com/engine/install/linux-postinstall/ Docker, Docker daemon attack surface: https://docs.docker.com/engine/security/#docker-daemon-attack-surface Docker, dockerd configuration reload behavior: https://docs.docker.com/reference/cli/dockerd/#configuration-reload-behavior moby, pkg/authorization/middleware.go: https://github.com/moby/moby/blob/b1bf272a8c767e2a21737fa0625844e9970c3dc2/pkg/authorization/middleware.go moby, daemon/pkg/plugin/backend_linux.go: https://github.com/moby/moby/blob/b1bf272a8c767e2a21737fa0625844e9970c3dc2/daemon/pkg/plugin/backend_linux.go moby, daemon/command/daemon.go: https://github.com/moby/moby/blob/b1bf272a8c767e2a21737fa0625844e9970c3dc2/daemon/command/daemon.go moby, issue #31836 (a disabled authz plugin blocking requests): https://github.com/moby/moby/issues/31836 moby, commit 38de272bd4 (removing a disabled authz plugin from the chain): https://github.com/moby/moby/commit/38de272bd4 moby, PR #33655 (Eliminate authz plugin disable race): https://github.com/moby/moby/pull/33655 NVD, CVE-2024-41110: https://nvd.nist.gov/vuln/detail/CVE-2024-41110 moby, GHSA-v23v-6jw2-98fq (CVE-2024-41110 advisory): https://github.com/moby/moby/security/advisories/GHSA-v23v-6jw2-98fq NVD, CVE-2026-34040: https://nvd.nist.gov/vuln/detail/CVE-2026-34040 NVD, CVE-2026-33997: https://nvd.nist.gov/vuln/detail/CVE-2026-33997 opa-docker-authz, main.go: https://github.com/open-policy-agent/opa-docker-authz/blob/v0.9/main.go opa-docker-authz, releases: https://github.com/open-policy-agent/opa-docker-authz/releases opa-docker-authz v0.9, go.mod: https://github.com/open-policy-agent/opa-docker-authz/blob/v0.9/go.mod Docker Hub, openpolicyagent/opa-docker-authz-v2 tags: https://hub.docker.com/r/openpolicyagent/opa-docker-authz-v2/tags OPA, v0 → v1 upgrade guide: https://www.openpolicyagent.org/docs/v0-upgrade OPA, v0.59.0 release notes (rego.v1 import): https://github.com/open-policy-agent/opa/releases/tag/v0.59.0 OPA, errors and strict built-in errors: https://www.openpolicyagent.org/docs/errors ","permalink":"https://onurbaydas.online/posts/docker-authz-opa-en/","summary":"\u003cp\u003e\u003cem\u003eBu yazının Türkçesi: \u003ca href=\"https://onurbaydas.online/posts/docker-authz-opa/\"\u003eOPA ile Docker Authorization\u003c/a\u003e\u003c/em\u003e\u003c/p\u003e\n\u003ch2 id=\"1-introduction\"\u003e1. Introduction\u003c/h2\u003e\n\u003cp\u003eI set up \u003ccode\u003eopa-docker-authz\u003c/code\u003e to restrict Docker on shared, multi-user Linux workstations. After a routine policy update, I noticed that \u003ccode\u003e--privileged\u003c/code\u003e containers were silently running. This article summarizes that incident and what I found in the source code afterward.\u003c/p\u003e\n\u003cp\u003eDocker authorization plugins let you evaluate every API request sent to the Docker daemon against a policy before the daemon processes it. \u003ccode\u003eopa-docker-authz\u003c/code\u003e is a plugin that implements this mechanism with OPA (Open Policy Agent) and its policy language, Rego.\u003c/p\u003e","title":"Docker Authorization with OPA: The Silent Fail-Open and Other Production Lessons"},{"content":"I\u0026rsquo;m Onur Baydaş, a systems and infrastructure engineer based in Turkey. Since 2024 I\u0026rsquo;ve been designing, running, and troubleshooting on-premises infrastructure: the kind of environments where there is no managed cloud service to fall back on and every layer, from storage to scheduling, is your responsibility.\nWhat I work on Virtualization and storage: Proxmox VE clusters with Ceph, VMware vCenter/ESXi, hyperconverged platforms, TrueNAS/ZFS. GPU infrastructure: NVIDIA GPU servers for AI and research workloads, GPU partitioning with MIG and vGPU, job scheduling with Slurm. Multi-user platforms: Shared Linux systems for many users: user isolation, container access control, and day-to-day operations. Container security: Restricting Docker on shared machines, authorization policies with OPA/Rego, and the gaps those policies can\u0026rsquo;t close. Identity and operations: Samba-based Active Directory, Kerberos/NFS home directories, centralized logging, and backup monitoring. What I write about I write about problems I actually hit in production: the silent failures, the misleading documentation, and what the source code says when the docs don\u0026rsquo;t. Every post is based on a real incident, tested on a live system, and backed by links to the source.\nWriting work I\u0026rsquo;m open to freelance technical writing on infrastructure, Linux, virtualization, container security, and GPU/AI infrastructure topics.\nContact Email: onurbaydas@gmail.com GitHub: github.com/onurbaydas I\u0026rsquo;m also finishing my degree in computer engineering.\n","permalink":"https://onurbaydas.online/en/about/","summary":"\u003cp\u003eI\u0026rsquo;m Onur Baydaş, a systems and infrastructure engineer based in Turkey. Since 2024\nI\u0026rsquo;ve been designing, running, and troubleshooting on-premises infrastructure: the\nkind of environments where there is no managed cloud service to fall back on and\nevery layer, from storage to scheduling, is your responsibility.\u003c/p\u003e\n\u003ch2 id=\"what-i-work-on\"\u003eWhat I work on\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e\u003cstrong\u003eVirtualization and storage:\u003c/strong\u003e Proxmox VE clusters with Ceph, VMware vCenter/ESXi,\nhyperconverged platforms, TrueNAS/ZFS.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eGPU infrastructure:\u003c/strong\u003e NVIDIA GPU servers for AI and research workloads, GPU\npartitioning with MIG and vGPU, job scheduling with Slurm.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eMulti-user platforms:\u003c/strong\u003e Shared Linux systems for many users: user isolation,\ncontainer access control, and day-to-day operations.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eContainer security:\u003c/strong\u003e Restricting Docker on shared machines, authorization\npolicies with OPA/Rego, and the gaps those policies can\u0026rsquo;t close.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eIdentity and operations:\u003c/strong\u003e Samba-based Active Directory, Kerberos/NFS home\ndirectories, centralized logging, and backup monitoring.\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"what-i-write-about\"\u003eWhat I write about\u003c/h2\u003e\n\u003cp\u003eI write about problems I actually hit in production: the silent failures, the\nmisleading documentation, and what the source code says when the docs don\u0026rsquo;t.\nEvery post is based on a real incident, tested on a live system, and backed by\nlinks to the source.\u003c/p\u003e","title":"About"}]