--- title: "My Debian web server build" date: "2024-10-17" path: "/articles/223-my-debian-web-server-build.html" section: "/articles" description: "The checklist I follow to turn a fresh Debian or Ubuntu VM into a PHP web server, as it evolved from 2017 to 2024: base hardening, Virtualmin or plain Nginx, several PHP versions side by side, send-only Postfix, a default-drop firewall, certificates and backups." --- I've kept a text file of the steps I follow to build a web server for most of my working life. The 2017 version was written for Debian 8 on a VPS and assumed Apache, Virtualmin, PHP 5 and a lot of things that have since gone away. The 2024 version is the same skeleton on Debian 12 (or Ubuntu LTS on the big cloud providers), with PHP 7.4 to 8.3 installed side by side and a firewall that drops by default. This is the current one, with notes where the older way still turns up on servers I'm asked to look after. The examples use `web1.example.com` on `192.0.2.10`, with the office on `203.0.113.10`. Run everything as root unless I say otherwise. ## 1. First login Some minimal images, and a few providers' "hybrid" products, come without an SSH server. If you're on the provider's console: ```bash apt-get install openssh-server systemctl enable --now ssh ``` Before doing anything else, make the shell history useful. Timestamps on history have saved me more than once when working out what someone did to a server and when: ```bash cat >> ~/.bashrc <<'EOF' export HISTTIMEFORMAT="%d/%m/%y %T " export HISTSIZE=10000 export HISTFILESIZE=10000 EOF source ~/.bashrc ``` Set the time zone, bring everything up to date, and give the machine its proper name: ```bash timedatectl set-timezone Europe/London apt-get update && apt-get -y dist-upgrade hostnamectl set-hostname web1 ``` `/etc/hosts` needs the fully qualified name against the server's own address. Postfix, Virtualmin and plenty of other things read it, and a server that can't resolve its own name causes odd problems later: ```text 127.0.0.1 localhost 192.0.2.10 web1.example.com web1 ``` Reboot now, while there's nothing on the box to lose, so you know it comes back up on the new kernel with the new name: ```bash shutdown -r now ``` (2017: `dpkg-reconfigure tzdata`, `nano /etc/hostname` and the `hostname` command. They still work.) ## 2. A user, SSH and updates Cloud images usually give you a default user with sudo and no root password. On a VPS you often get root with a password. Either way, end up with a named admin user, key-only SSH and no password logins: ```bash adduser admin usermod -aG sudo admin mkdir -p /home/admin/.ssh cp ~/.ssh/authorized_keys /home/admin/.ssh/ chown -R admin:admin /home/admin/.ssh chmod 700 /home/admin/.ssh && chmod 600 /home/admin/.ssh/authorized_keys ``` On a cloud image, that user is also what you'll log in to Webmin with later, since root has no password there. Give it a password with `passwd admin` for sudo and the panel; SSH will still only accept keys. Then `/etc/ssh/sshd_config.d/local.conf`: ```text PermitRootLogin no PasswordAuthentication no KbdInteractiveAuthentication no ``` ```bash sshd -t && systemctl reload ssh ``` Open a second session as `admin` before you close the first. If the second one can't get in, you still have the first. Security updates should install themselves: ```bash apt-get install unattended-upgrades dpkg-reconfigure -plow unattended-upgrades ``` Time sync: in 2017 I installed `ntpdate` and pointed it at the UK pool servers (`0.uk.pool.ntp.org` to `3.uk.pool.ntp.org`). Debian now ships `systemd-timesyncd`, which is enough for a web server; check it's on with `timedatectl` and look for `System clock synchronized: yes`. Some general tools I always want: ```bash apt-get -y install zip unzip screen curl plocate git net-tools ``` ## 3. Virtualmin, or plain Nginx Most of my servers host several sites for different people, and for that I use Virtualmin GPL: it creates a Unix user, home directory, PHP-FPM pool, database and web server config per site, and handles certificates and backups. When a server runs one application, I skip it and configure Nginx by hand. Both are below. ### With Virtualmin Run the installer inside `screen`, so a dropped SSH session doesn't kill it halfway: ```bash screen curl -fsSLo virtualmin-install.sh https://software.virtualmin.com/gpl/scripts/virtualmin-install.sh sh virtualmin-install.sh --bundle LEMP --minimal ``` `--bundle LEMP` installs Nginx rather than Apache, and `--minimal` leaves out the mail filtering, antivirus and other services a web-only server doesn't need. In 2017 there were no such options: you installed the full Apache stack, stopped Apache, installed `webmin-virtualmin-nginx` and `webmin-virtualmin-nginx-ssl`, and then turned features off one by one. Log in at `https://192.0.2.10:10000` and run the post-install wizard. Then, under System Settings, Features and Plugins, turn off everything you won't use. On a web-only box that's typically the DNS domain, mail, spam and virus filtering, Mailman, AWstats, DAV and protected directories. Do that before removing packages, or Virtualmin's configuration check fails because it expects them. Remove the installer and its log afterwards; the log contains more about the box than you want lying around: ```bash rm -f ~/virtualmin-install.sh ~/virtualmin-install.log ``` ### Without it ```bash apt-get install nginx mariadb-server ``` Then one PHP-FPM pool and one Nginx server block per site, set up as in the next section. ### Clearing out Whichever route, remove services that aren't needed rather than just firewalling them. On a web-only server: ```bash apt-get purge --auto-remove bind9 dovecot-core spamassassin clamav clamav-daemon clamav-freshclam mailman postgresql rpcbind ``` `rpcbind` is easy to miss; it only matters for NFS and listens on port 111 on every interface by default. Check what's left afterwards with `ss` (section 6). ## 4. Several PHP versions side by side Sites move to new PHP versions at their own pace, so a shared server needs several installed at once. Ondřej Surý's packages make that easy: each version installs under its own name (`php8.2-fpm`, `php7.4-fpm` and so on) with its own config in `/etc/php//`. On Ubuntu it's a PPA: ```bash add-apt-repository ppa:ondrej/php apt-get update ``` On Debian, add the repository and its signing key: ```bash apt-get install -y lsb-release ca-certificates curl curl -sSLo /usr/share/keyrings/deb.sury.org-php.gpg https://packages.sury.org/php/apt.gpg echo "deb [signed-by=/usr/share/keyrings/deb.sury.org-php.gpg] https://packages.sury.org/php/ $(lsb_release -sc) main" > /etc/apt/sources.list.d/php.list apt-get update ``` Then the same set of extensions for each version you need: ```bash for v in 7.4 8.1 8.2 8.3; do apt-get -y install php$v-fpm php$v-cli php$v-mysql php$v-gd php$v-curl php$v-intl \ php$v-mbstring php$v-zip php$v-xml php$v-soap php$v-bcmath done ``` `php-mcrypt` is in all my older lists and is gone from PHP 7.2 onwards; don't go looking for it. Choose which version the plain `php` command runs, for cron jobs and Composer: ```bash update-alternatives --set php /usr/bin/php8.2 ``` ### Pools Each site gets its own FPM pool running as the site's own user, so one compromised site can't read another's files. Virtualmin creates these for you and lets you pick the version per site. By hand, `/etc/php/8.2/fpm/pool.d/example.com.conf`: ```text [example.com] user = example group = example listen = /run/php/php8.2-fpm-example.sock listen.owner = www-data listen.group = www-data pm = ondemand pm.max_children = 10 php_admin_value[error_log] = /home/example/logs/php-error.log ``` And in the site's Nginx server block: ```nginx location ~ \.php$ { include snippets/fastcgi-php.conf; fastcgi_pass unix:/run/php/php8.2-fpm-example.sock; } ``` Moving a site to another PHP version is a matter of moving the pool file to the other version's `pool.d` and changing the socket path. ### Settings My old lists edited `php.ini` for each version and then deleted the CLI and CGI copies and symlinked them to the Apache one, so every SAPI used the same settings. It works, but upgrades fight you over it. A cleaner way is one small file per version that `phpenmod` links into every SAPI. `/etc/php/8.2/mods-available/local.ini`: ```text ; priority=99 expose_php = Off max_execution_time = 300 max_input_time = 600 memory_limit = 512M session.gc_probability = 1 opcache.enable = 1 opcache.memory_consumption = 256 ``` ```bash phpenmod -v 8.2 local systemctl reload php8.2-fpm ``` `expose_php = Off` stops PHP announcing its version in an `X-Powered-By` header. The time and memory limits are what the CMSs I host need; set your own. `session.gc_probability = 1` needs explaining. Debian turns PHP's own session clean-up off and runs a cron job instead, which only knows about the default session directory. Virtualmin gives each site its own session directory, which that cron job never sees, so old sessions pile up until the disk fills. Turning PHP's garbage collection back on fixes it. If a site uses code encoded with ionCube, the loader comes from ionCube's website as a zip containing one `.so` per PHP version. Unpack it to `/usr/local/ioncube` and add `zend_extension = /usr/local/ioncube/ioncube_loader_lin_8.2.so` (matching the version) to that version's `local.ini`, before any other Zend extensions. If nothing needs it, don't install it. ## 5. Send-only Postfix Web applications need to send mail: password resets, order confirmations, cron output. The server should never receive it. Postfix as a null client does that. In `/etc/postfix/main.cf`, replace the existing `inet_interfaces` line and add the rest: ```text inet_interfaces = loopback-only inet_protocols = ipv4 mydestination = local_transport = error:local delivery is disabled ``` `loopback-only` means Postfix only accepts mail from programs on the server itself. `local_transport` stops it delivering anything locally, so a mistake can't leave mail piling up in `/var/mail`. `inet_protocols = ipv4` is there because many hosts have no reverse DNS on their IPv6 addresses, and big mail providers reject mail from addresses without it. In `/etc/postfix/master.cf`, comment out the submission service if it's enabled; nobody should be authenticating to this server to send mail: ```text #submission inet n - y - - smtpd ``` Cron and system mail goes to `root@` the server's domain. Send it somewhere someone reads with `/etc/postfix/virtual`: ```text root you@example.com ``` ```bash postconf -e 'virtual_alias_maps = hash:/etc/postfix/virtual' postmap /etc/postfix/virtual ``` Sending straight to the internet from a VPS address often lands in spam, and some providers block port 25 outright. Then it's better to relay through your provider's relay or a transactional mail service: ```text relayhost = [smtp.example.com]:587 smtp_sasl_auth_enable = yes smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd smtp_sasl_security_options = noanonymous smtp_tls_security_level = encrypt ``` `/etc/postfix/sasl_passwd`: ```text [smtp.example.com]:587 your-username:your-password ``` ```bash chmod 600 /etc/postfix/sasl_passwd postmap /etc/postfix/sasl_passwd systemctl restart postfix ``` Whichever way it leaves, the sending domain needs SPF and DKIM records covering it, or receivers will treat it as forged. ## 6. Check what's listening This is the step I'd keep if I could only keep one. After all of the above: ```bash ss -tulpn ``` (In 2017 it was `netstat -tulpn`; that's in `net-tools` and still works.) On a Virtualmin web server, what you want to see is roughly: ```text Netid State Local Address:Port Process tcp LISTEN 127.0.0.1:3306 mariadbd tcp LISTEN 127.0.0.1:25 master tcp LISTEN 0.0.0.0:22 sshd tcp LISTEN 0.0.0.0:80 nginx tcp LISTEN 0.0.0.0:443 nginx tcp LISTEN 0.0.0.0:10000 miniserv.pl tcp LISTEN 0.0.0.0:20000 miniserv.pl ``` MySQL and Postfix on loopback only, the web server and SSH on everything, and Webmin (10000) and Usermin (20000). Anything else listening on `0.0.0.0` needs a reason. ## 7. Saying less about yourself Version numbers in banners and headers help attackers pick an exploit. Turn them off. Nginx, in `/etc/nginx/nginx.conf`: ```nginx server_tokens off; ``` Apache, if you're on it, in `/etc/apache2/conf-available/security.conf`: ```apache ServerTokens Prod ServerSignature Off TraceEnable Off ``` On Apache I also turn off the modules nothing uses and make sure the ones every CMS expects are on: ```bash a2dismod -f autoindex dav dav_fs a2enmod rewrite headers expires systemctl restart apache2 ``` If ProFTPD is installed (older Virtualmin installs always had it), add `ServerIdent off` under `ServerName` in `/etc/proftpd/proftpd.conf`. Better still, don't run FTP at all; SFTP comes with SSH. Admin tools under a site, such as phpMyAdmin, should only answer the office. Nginx: ```nginx location /dbadmin { allow 203.0.113.10; deny all; } ``` Apache 2.4: ```apache Require ip 203.0.113.10 ``` The `Order deny,allow` / `Allow from` syntax in older configs is Apache 2.2. It only works on 2.4 through `mod_access_compat`, and mixing the two styles gives results you won't expect. ## 8. Firewall: drop by default Every version of my list has had an iptables ruleset, but the early ones had the INPUT policy set to ACCEPT and sent a list of admin ports to a chain that only allowed the office. That protects the ports on the list and nothing else: install Elasticsearch or Redis on a public interface next month and it's open to the world. The current version drops everything not explicitly allowed. On Debian 12 I write it in nftables directly. `/etc/nftables.conf`: ```text #!/usr/sbin/nft -f flush ruleset table inet filter { set admin_v4 { type ipv4_addr flags interval elements = { 203.0.113.10, 198.51.100.0/28 } } chain input { type filter hook input priority 0; policy drop; iif lo accept ct state established,related accept ct state invalid drop ip protocol icmp icmp type { echo-request, destination-unreachable, time-exceeded } accept meta l4proto ipv6-icmp accept tcp dport { 80, 443 } accept ip saddr @admin_v4 tcp dport { 22, 10000, 20000 } accept } chain forward { type filter hook forward priority 0; policy drop; } chain output { type filter hook output priority 0; policy accept; } } ``` ```bash nft -c -f /etc/nftables.conf systemctl enable --now nftables nft list ruleset ``` `nft -c` checks the file without loading it. The `inet` table covers IPv4 and IPv6 together, which iptables never did; plenty of servers with a careful iptables ruleset are wide open over IPv6. The admin set holds the office and a VPN range; add a v6 set the same way if you connect over IPv6. Keep the provider's console login working, because one day you'll lock yourself out. The same thing in iptables, for older servers, goes in `/etc/iptables/rules.v4` with `iptables-persistent` installed: ```text *filter :INPUT DROP [0:0] :FORWARD DROP [0:0] :OUTPUT ACCEPT [0:0] :ADMIN - [0:0] -A INPUT -i lo -j ACCEPT -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT -A INPUT -m conntrack --ctstate INVALID -j DROP -A INPUT -p icmp -j ACCEPT -A INPUT -p tcp -m multiport --dports 80,443 -j ACCEPT -A INPUT -p tcp -m multiport --dports 22,10000,20000 -j ADMIN -A ADMIN -s 203.0.113.10/32 -j ACCEPT -A ADMIN -s 198.51.100.0/28 -j ACCEPT -A ADMIN -j DROP COMMIT ``` Load it with `iptables-apply /etc/iptables/rules.v4`, which rolls back if you don't confirm in time. If you have an iptables file already, `iptables-restore-translate -f rules.v4` produces nftables syntax as a starting point. Two notes. Newer Virtualmin installers may set up firewalld for you; if so, use that (zones and sources do the same job) or remove it, but don't run two firewall managers. And Virtualmin installs fail2ban, which adds its own rules on top; that's fine with either. ## 9. Certificates With Virtualmin, request a Let's Encrypt certificate for each site from its SSL settings page. It renews them itself, and the same page can copy a site's certificate to Webmin, Usermin and Postfix so the admin panel stops showing a browser warning. Without it: ```bash apt-get install certbot python3-certbot-nginx certbot --nginx -d example.com -d www.example.com certbot renew --dry-run ``` Debian's package installs a systemd timer for renewal; `systemctl list-timers | grep certbot` shows it. The dry run proves renewal will work before the first one is due in two months. Back in 2017 I terminated TLS in Pound in front of Varnish, and Pound wanted the key, certificate and chain in a single file (`cat ssl.key ssl.cert ssl.ca > ssl.pem`). If you meet that setup, that file has the private key in it, so it needs the same permissions as the key. ## 10. The database On current Debian and Ubuntu, MariaDB's and MySQL's `root` account authenticates by Unix socket: `mysql` as the system root user gets straight in with no password, and nobody else can log in as root at all. Leave it that way. Older lists, including mine, had a step to switch root back to password authentication with an `UPDATE mysql.user` statement; that's both less secure and, on MySQL 8, broken, because `PASSWORD()` no longer exists. For your own access, create a named admin user: ```sql CREATE USER 'dbadmin'@'localhost' IDENTIFIED BY 'CHANGE_ME'; GRANT ALL PRIVILEGES ON *.* TO 'dbadmin'@'localhost' WITH GRANT OPTION; ``` And put its credentials in `~/.my.cnf` with `chmod 600`, rather than typing them on the command line where they land in the history: ```text [client] user=dbadmin password=CHANGE_ME ``` `mytop` and the MySQL clients read that file. MySQLTuner is still worth running after a week of real traffic: ```bash apt-get install mysqltuner mytop mysqltuner ``` For Magento sites, Redis goes on the same box (`apt-get install redis-server`). Debian's package binds it to localhost only. Check that it does, because Redis on a public interface with no password is one of the most common ways servers get taken over. ## 11. Backups A backup you haven't restored isn't a backup. What I do: - Virtualmin's scheduled backups (Backup and Restore, Scheduled Backups) of every site: files, databases and Virtualmin's own settings. Daily, to a local directory and to an S3 bucket. - Without Virtualmin, a nightly cron job that runs `mysqldump --single-transaction` for each database and `tar` for each site, then copies the results off the server. - The off-site copy goes to a bucket used for nothing else, with versioning on and a lifecycle rule to expire old versions. The credentials on the server can only write to that bucket. On AWS, use an instance role instead of keys. Elsewhere, an access key limited to that bucket, stored with `chmod 600`. - Every few months, restore one site onto a scratch server and check it loads. In 2017 the off-site copy went up with `s3cmd` from the s3tools APT repository. These days `s3cmd` is in Debian's own archive, and the AWS CLI does the same job. ## 12. Finishing ```bash updatedb shutdown -r now ``` `updatedb` builds the `locate` index so `locate` works straight away. The final reboot proves everything starts on its own: the firewall loads, PHP-FPM pools come up, the sites answer. Better to find out now than at 3am after a kernel update. My 2017 list ended with `history -c`. I put it there because I'd typed passwords on the command line during the build. The better fix is not to: use `~/.my.cnf`, `read -s`, or a password prompt, and leave the history alone. It's the record of what was done to the server.