--- title: "Load testing a site from its sitemap with Siege" date: "2020-11-03" path: "/207-load-testing-a-site-from-its-sitemap-with-siege.html" section: "/ramblings" description: "Building a URL list from a site's XML sitemap and using Siege to load test it or warm its cache, with notes on reading the results." --- Hitting the home page a thousand times tells you how fast the home page is when it's cached. Real visitors land on category pages, product pages and old blog posts, many of which aren't. The quickest way to get a realistic list of URLs is the XML sitemap the site already publishes, and Siege will work through a list like that happily. I first wrote this up for [Magento stores in 2012](/82-magento-siege-testing.html); I still use the same approach, but the URL extraction needed redoing. Only do this to sites you're responsible for, and tell the hosting provider first if it isn't your own server. From the outside a load test looks exactly like a denial of service attack. ## Install ```bash apt-get install siege siege --version ``` Run it from a machine other than the web server, ideally one with a decent connection. Running it on the server itself means Siege and the site compete for the same CPU. ## Build the URL list My old version split the sitemap on `` tags with `sed` and kept lines whose priority was 0.5 or 1.0, which matched how Magento 1 laid out its sitemap. It broke on any sitemap that didn't use those exact priorities or put everything on one line. Pulling out every `` is simpler and works on any sitemap: ```bash curl -s https://www.example.com/sitemap.xml \ | grep -oP '(?<=)[^<]+' > urls.txt wc -l urls.txt ``` `grep -o` prints only the matching part, and the `(?<=)` lookbehind (Perl regex, hence `-P`) starts the match just after the opening tag. Many sites now publish a sitemap index instead, a sitemap of sitemaps. WordPress 5.5 does this at `/wp-sitemap.xml`, and SEO plugins and Magento do it on larger sites. Fetch each child sitemap in turn: ```bash curl -s https://www.example.com/wp-sitemap.xml \ | grep -oP '(?<=)[^<]+' \ | while read -r map; do curl -s "$map" | grep -oP '(?<=)[^<]+' done > urls.txt ``` If the sitemap is gzipped (`sitemap.xml.gz`), add `--compressed` to `curl`, or pipe it through `gunzip`. A big store can give you tens of thousands of URLs. For a load test a random sample is enough: ```bash shuf -n 500 urls.txt > sample.txt ``` ## Run the test ```bash siege -c 50 -t 120S -i -f sample.txt ``` - `-c 50` runs 50 simulated users at once. - `-t 120S` runs for 120 seconds. The unit is required: `S`, `M` or `H`. - `-i` ("internet" mode) makes each user pick URLs at random from the file instead of going through it in order, which is closer to real traffic. - `-f` is the URL file. By default each simulated user waits a random delay of up to a second between requests (`-d` changes it). Add `-b` for benchmark mode, which drops the delay and sends requests as fast as possible. That's useful for finding the ceiling, less so for modelling real visitors. Siege prints the summary on standard error, so `siege ... > result.txt` gives you an empty file. Capture both: ```bash siege -c 50 -t 120S -i -f sample.txt 2>&1 | tee siege-result.txt ``` ## Reading the summary The figures that matter: - Availability: the percentage of requests that succeeded. Anything under 100% needs explaining before you look at anything else. - Response time: the average seconds per request. - Transaction rate: requests per second the site sustained. - Longest transaction: the slowest single request. A big gap between this and the average usually means some pages are uncached or a database query is slow. - Failed transactions: responses with a status of 400 or above, plus socket errors and timeouts. Use `-v` on a short run to see which URLs are failing and why. Start low and raise `-c` in steps (10, 25, 50, 100), running each for a couple of minutes. The interesting number is the concurrency where response time stops being flat and starts climbing. While it runs, watch the server with `top` or `htop`, and if there's a cache in front, `varnishstat` shows whether requests are being served from cache. ## Cache warming The same list makes a good cache warmer after a deploy or a full cache flush. One user, working through every URL in order, with each request shown: ```bash siege -c 1 -r 1 -v -f urls.txt ``` `-r 1` means one pass through the file and then stop. `-v` prints each request with its status code and time, so you can also spot any URLs in the sitemap that return 404 or redirect, which shouldn't be in a sitemap at all. ## Settings file Siege keeps its defaults in `~/.siege/siege.conf` (run `siege.config` to create one). Two lines are worth checking before a test: `accept-encoding`, so Siege asks for compressed pages the way a browser does, and `user-agent`, so you can pick out the test traffic in the access log afterwards. Extra request headers can also be added for a single run with `-H`: ```bash siege -c 25 -t 60S -i -H "Accept-Encoding: gzip" -f sample.txt ```