---
title: "Recording IQ with SigMF metadata"
date: "2022-12-30"
path: "/219-recording-iq-with-sigmf-metadata.html"
section: "/ramblings"
description: "What the Signal Metadata Format is, why raw IQ files need it, and a short Python example that writes a recording with its metadata and reads it back."
---

After a few years of SDR work I had a disk full of files called things like `capture_433M_2Msps.bin`. The file name was the only record of the centre frequency and sample rate, and nothing at all recorded the sample format, the gain, the hardware or what was being transmitted at the time. Opening an old one meant guessing whether it was 8-bit unsigned from an RTL-SDR, 16-bit integers from a USRP or 32-bit floats from GNU Radio.

SigMF, the Signal Metadata Format, fixes that with a JSON sidecar file. It's an open specification, GNU Radio and a growing number of tools understand it, and it's simple enough to write by hand.

## The format

A SigMF recording is two files with the same base name:

- `name.sigmf-data`: the raw samples, exactly as they came off the radio. No header, no conversion.
- `name.sigmf-meta`: JSON describing them.

The two can also be bundled into a single `name.sigmf` tar archive for sharing.

The metadata has three sections:

- `global`: facts about the whole file. The required ones are `core:datatype` and `core:version`; `core:sample_rate` is optional in the spec but you always want it. Hardware, author, description and licence go here too.
- `captures`: a list of segments, each starting at a sample index. Each says what the radio was tuned to (`core:frequency`) and when it started (`core:datetime`). A recording made in one go has one capture; if you retuned mid-recording you'd add another at the sample where it changed.
- `annotations`: optional notes on ranges of samples, each with a start, a length and, if you like, frequency edges and a comment. Useful for marking where a test transmission is in a long file.

Keys are namespaced. Everything in the standard is `core:`, and extensions add their own namespaces, so tools can ignore anything they don't understand.

## Datatypes

`core:datatype` is the bit that saves the guessing. It's built from:

- `c` for complex (interleaved I and Q) or `r` for real,
- `f`, `i` or `u` for float, signed or unsigned integer,
- the bit width,
- `_le` or `_be` for byte order (not needed for 8-bit types).

So the common ones are:

| Source | Raw format | SigMF datatype |
|--------|------------|----------------|
| `rtl_sdr` | 8-bit unsigned I/Q | `cu8` |
| `hackrf_transfer` | 8-bit signed I/Q | `ci8` |
| UHD `--type short` | 16-bit signed I/Q | `ci16_le` |
| GNU Radio complex, NumPy `complex64` | 32-bit float I/Q | `cf32_le` |

Because the data file is untouched, you can describe an existing raw capture without converting it, just by writing the right metadata next to it.

## Writing a recording in Python

The `sigmf` package on PyPI is the reference library. It computes a SHA-512 of the data file, validates the metadata against the schema and writes the JSON:

```bash
python3 -m pip install sigmf numpy
```

This example generates a test tone in place of real radio samples, so it runs anywhere. In practice `samples` comes from your SDR's API, or the data file already exists from a command-line capture.

```python
import numpy as np
import sigmf
from sigmf.utils import get_data_type_str, get_sigmf_iso8601_datetime_now

sample_rate = 2_000_000
centre_freq = 433_920_000
n = sample_rate // 10

t = np.arange(n) / sample_rate
samples = (0.5 * np.exp(2j * np.pi * 100_000 * t)).astype(np.complex64)
samples.tofile("tone.sigmf-data")

meta = sigmf.SigMFFile(
    data_file="tone.sigmf-data",
    global_info={
        sigmf.DATATYPE_KEY: get_data_type_str(samples),
        sigmf.SAMPLE_RATE_KEY: sample_rate,
        sigmf.HW_KEY: "RTL-SDR v3",
        sigmf.DESCRIPTION_KEY: "100 kHz test tone from a signal generator",
    },
)
meta.add_capture(0, metadata={
    sigmf.FREQUENCY_KEY: centre_freq,
    sigmf.DATETIME_KEY: get_sigmf_iso8601_datetime_now(),
})
meta.add_annotation(0, n, metadata={
    sigmf.FREQ_LOWER_EDGE_KEY: centre_freq + 95_000,
    sigmf.FREQ_UPPER_EDGE_KEY: centre_freq + 105_000,
    sigmf.COMMENT_KEY: "test tone, -40 dBm at the generator",
})
meta.tofile("tone.sigmf-meta")
```

`get_data_type_str()` works the datatype out from the NumPy array (`cf32_le` here), which avoids getting it wrong by hand. `tofile()` validates before writing, so a missing required field fails here and not months later when someone tries to open the file.

The resulting `tone.sigmf-meta`:

```json
{
    "global": {
        "core:datatype": "cf32_le",
        "core:description": "100 kHz test tone from a signal generator",
        "core:hw": "RTL-SDR v3",
        "core:num_channels": 1,
        "core:offset": 0,
        "core:sample_rate": 2000000,
        "core:sha512": "0d26...2c03",
        "core:version": "1.2.6"
    },
    "captures": [
        {
            "core:datetime": "2022-12-30T14:02:11.418263Z",
            "core:frequency": 433920000,
            "core:sample_start": 0
        }
    ],
    "annotations": [
        {
            "core:comment": "test tone, -40 dBm at the generator",
            "core:freq_lower_edge": 434015000,
            "core:freq_upper_edge": 434025000,
            "core:sample_count": 200000,
            "core:sample_start": 0
        }
    ]
}
```

I've shortened the hash. The `core:version` is whichever version of the spec your installed library writes.

## Reading it back

```python
import sigmf

rec = sigmf.fromfile("tone")
print(rec.sample_rate)
print(rec.get_captures()[0][sigmf.FREQUENCY_KEY])

samples = rec.read_samples()
first = rec[0:1024]
```

`fromfile` finds the `.sigmf-meta` and `.sigmf-data` from the base name, or opens a `.sigmf` archive. `read_samples()` returns a complex NumPy array whatever the stored datatype, so code that reads `cu8` from an RTL-SDR and `ci16_le` from a USRP looks the same. Slicing reads just that range from disk, which matters when the file is several gigabytes.

To produce a single file for sharing, write the archive instead:

```python
meta.tofile("tone.sigmf")
```

## What to watch for

- Write the metadata at the start of a capture, not afterwards from memory. The datetime and frequency should come from the same code that tuned the radio.
- Record the gain and the antenna in the description or an extension field. They're the first things you'll want when comparing two recordings.
- Interleaved 8-bit RTL-SDR samples are unsigned and centred around 128, not zero. Declare them as `cu8` and let the library convert them, rather than converting yourself and forgetting the offset.
- The library hashes the whole data file when it writes the metadata. On a multi-gigabyte capture that takes a noticeable time, so do it after the capture has finished, not in the receive loop.

Today: older releases of the library kept the key constants on the class (`SigMFFile.DATATYPE_KEY` and so on). They still work, with deprecation warnings, but new code should use the module-level names as above.