---
title: "NFS and Samba mounts between Linux servers"
date: "2016-11-02"
path: "/181-nfs-and-samba-mounts-between-linux-servers.html"
section: "/ramblings"
description: "Sharing a CMS uploads directory between two Debian web servers over NFSv4, with the ownership and troubleshooting steps that cost me most of an afternoon, plus a CIFS mount for when the other end is Samba."
---

The job: a CMS ran on two Debian web servers, and the uploaded files directory had to be the same on both. Anything an editor uploaded on one box had to be visible on the other straight away. The simplest answer on a private network is to keep the files on one server and mount them on the other over NFS.

In the examples the file server is `192.0.2.10`, the second web server is `192.0.2.20`, and the site lives in `/home/example/public_html` owned by the user `example`. On a real setup these should be addresses on the private interface between the two machines, not the public ones.

## The file server

Install the server side:

```bash
apt-get install nfs-kernel-server
```

I don't export the site directory directly. NFSv4 likes a single export root, so I make `/export` the root and bind-mount the real directory into it. That keeps the export paths short and means nothing outside `/export` is ever reachable.

```bash
mkdir -p /export/uploads
mount --bind /home/example/public_html/application/files /export/uploads
```

Make the bind mount survive a reboot in `/etc/fstab`:

```text
/home/example/public_html/application/files  /export/uploads  none  bind  0  0
```

Then the exports, in `/etc/exports`:

```text
/export          192.0.2.20(rw,sync,fsid=0,no_subtree_check,crossmnt)
/export/uploads  192.0.2.20(rw,sync,nohide,no_subtree_check)
```

What the options mean:

- `fsid=0` marks `/export` as the NFSv4 pseudo-root. Clients then mount paths relative to it, so `/export/uploads` is mounted as `192.0.2.10:/uploads`.
- `nohide` and `crossmnt` let the client see the bind-mounted directory under the root instead of an empty folder.
- `sync` makes the server commit writes to disk before replying. `async` is faster, and I tried it, but a crash can lose writes the client thinks are saved. For files people upload once and expect to stay, `sync` is the right default.
- `no_subtree_check` turns off a check that causes more problems than it solves when files get renamed.
- Root squashing is on by default, which maps root on the client to `nobody` on the server. Leave it on. `no_root_squash` lets root on the web server do anything to the files, which you don't want on a box that runs PHP.

Load the exports and check them:

```bash
exportfs -rav
service nfs-kernel-server restart
showmount -e localhost
```

## Ownership: the part that wastes the afternoon

NFS passes numeric user and group ids around. If `example` is uid 1005 on the file server and uid 1003 on the web server, the web server's PHP processes can't write to the share, and `ls -l` shows the wrong owner or `nobody`.

Check both ends:

```bash
id example
```

If the numbers differ, fix one side so they match (`usermod -u` and `groupmod -g`, then chown the user's files), or create the user with an explicit id on the second server in the first place.

NFSv4 also maps names through `idmapd`. Both machines need the same domain in `/etc/idmapd.conf`:

```text
[General]
Domain = example.com
```

Restart `nfs-common` on the client and `nfs-kernel-server` on the server after changing it. If files still show as `nobody:nogroup` on the client, the domains don't match.

I went round the loop of `chmod 777` on the export directory before getting this right. Don't. Once the ids line up, normal ownership and `755`/`644` permissions work, and if you use Virtualmin you can put a site's permissions back with:

```bash
virtualmin fix-domain-permissions --domain example.com
```

## Locking it down

NFSv4 only needs TCP 2049. Allow that from the web server's address and nothing else. NFSv3 and `showmount` also use rpcbind on 111 and `mountd` on a random port, which is one more reason to stick to v4.

The TCP wrappers files still apply to rpcbind and mountd on this era of Debian, so as a second layer:

```text
# /etc/hosts.deny
rpcbind mountd : ALL

# /etc/hosts.allow
rpcbind mountd : 192.0.2.20 127.0.0.1
```

## The web server

```bash
apt-get install nfs-common
mkdir -p /home/example/public_html/application/files
mount -t nfs4 192.0.2.10:/uploads /home/example/public_html/application/files
```

Try creating and deleting a file as the site user, then make it permanent in `/etc/fstab`:

```text
192.0.2.10:/uploads  /home/example/public_html/application/files  nfs4  _netdev,noatime,bg,proto=tcp  0  0
```

`_netdev` stops the system trying to mount it before the network is up, and `bg` means a missing file server makes the boot carry on and retry in the background instead of hanging. I first had a long attribute cache (`actimeo=1800`) to cut down chatter, but that means a new upload can take up to half an hour to appear on the other server. For a shared uploads directory the defaults are the right trade.

To unmount:

```bash
umount /home/example/public_html/application/files
```

## When it doesn't work

These are the commands I ended up running over and over:

```bash
# is anything listening on 111 and 2049?
netstat -tulpn | grep -E ':(111|2049) '

# which RPC services are registered?
rpcinfo -p

# does the NFS service answer over TCP?
rpcinfo -t 192.0.2.10 nfs

# what does the server export? (needs rpcbind and mountd)
showmount -e 192.0.2.10
```

If `rpcinfo -p` fails on the server, rpcbind isn't running (`service rpcbind restart`). If it works locally but not from the client, it's the firewall or `hosts.allow`. If the mount works but writes fail, go back to the ownership section.

## When the other end is Samba

If the share lives on a Windows box or a Samba server instead, the client side is CIFS:

```bash
apt-get install cifs-utils
```

Keep the credentials out of `fstab` in a file only root can read, `/root/.smb-example`:

```text
username=example
password=your-password
```

```bash
chmod 600 /root/.smb-example
```

And the `fstab` line:

```text
//192.0.2.30/uploads  /mnt/uploads  cifs  credentials=/root/.smb-example,uid=example,gid=example,file_mode=0644,dir_mode=0755,vers=3.0,_netdev  0  0
```

CIFS doesn't carry Unix ownership, so `uid` and `gid` decide who owns everything on the mounted side. Set them to the site user and the web server can write to it.