---
title: "Recovering a NAS RAID with mdadm and LVM"
date: "2018-01-25"
path: "/185-recovering-a-nas-raid-with-mdadm-and-lvm.html"
section: "/ramblings"
description: "Pulling a disk out of a dead consumer NAS and reading its data on Linux by reassembling the Linux RAID and LVM layers by hand."
---

A consumer NAS had died, but the disks were fine. These boxes almost always run Linux underneath, laying the data out as a Linux software RAID (`md`) with LVM on top, so you can pull a drive, attach it to any Linux machine and read it directly. I did this from a Kali live system, but any recent Debian or Ubuntu will do.

The one rule throughout: mount everything **read only**. If the array is degraded you do not want the recovery host writing to it.

## Tools

```bash
apt-get install mdadm lvm2
modprobe dm-mod
```

## Look at what is on the disk

Plug the disk in and see the partitions:

```bash
lsblk
```

A NAS disk has a lot of small partitions (the firmware, swap, config and so on) and one big one for your data. `file -s` on the data partition confirms it is a RAID member:

```bash
file -s /dev/sdb10
```

```text
/dev/sdb10: Linux Software RAID version 1.2 (1) UUID=... level=1 disks=2
```

Now read the RAID superblock. The key field is **Data Offset**, which is how far into the partition the actual filesystem starts:

```bash
mdadm --examine /dev/sdb10
```

```text
/dev/sdb10:
        Version : 1.2
     Raid Level : raid1
   Raid Devices : 2
  Avail Dev Size : 3898521911 (1858.96 GiB 1996.04 GB)
    Data Offset : 2048 sectors
   Super Offset : 8 sectors
          State : clean
   Device Role : Active device 0
   Array State : A. ('A' == active, '.' == missing, 'R' == replacing)
```

That `A.` means device 0 is present and its mirror is missing. For a RAID 1 that is fine, as one half of a mirror is a complete copy. The data is all there.

## Option 1: let mdadm assemble it

The clean way is to let `mdadm` assemble the array from the member, force it to run even though it is degraded, then bring up the LVM volume group that lives inside it:

```bash
mdadm -A -R /dev/md9 /dev/sdb10
vgchange -ay
mkdir -p /mnt/recovery
mount /dev/<vg-name>/<lv-name> /mnt/recovery -o ro,user
```

`vgchange -ay` activates whatever volume group it finds; `lsblk` or `vgs` afterwards will show you the volume group and logical volume names to use in the mount.

## Option 2: skip mdadm and go straight to the data

Sometimes mdadm will not assemble the array (wrong metadata version, no matching mirror, a fussy live environment). Because this is RAID 1, the filesystem sits as a plain copy at the data offset, so you can expose it as a loop device and let LVM find it directly.

Take the Data Offset from `mdadm --examine` and multiply by the 512-byte sector size. For an offset of 2048 sectors that is `2048 * 512`:

```bash
losetup --find --show --read-only --offset $((2048*512)) /dev/sdb10
vgscan
vgchange -ay
mount /dev/<vg-name>/<lv-name> /mnt/recovery -o ro,user
```

Older firmware used a much smaller offset; on one older box it was 272 sectors, so always read the real value from the superblock rather than assuming.

## When the filesystem will not mount cleanly

If the kernel driver refuses the filesystem (a slightly nonstandard ext variant, or minor corruption), `fuseext2` will often mount it read only where the kernel will not:

```bash
fuseext2 -o ro -o sync_read /dev/mapper/<vg-name>-<lv-name> /mnt/recovery
```

## Get the data off

Once it is mounted, copy everything to a known-good disk and walk away. `rsync` with `--progress` lets you see it working and resume if it stops:

```bash
mkdir /backup
mount /dev/sdc1 /backup
rsync -a --progress /mnt/recovery/ /backup/
```

The whole job comes down to three layers in order: the partition, the `md` RAID on top of it, and the LVM volume inside that. Peel them back one at a time, keep it read only, and a dead NAS is just a disk full of files.