GNU radio companion raw captured data file split

Question

My file is a simple raw data 32 bit floats sampled at 384 kHz. No metadata or anything alike. The file is some 10+ GB. I am looking for a method of isolating a certain group of signals from it into a new raw data file. I know where the group starts and how long it lasts. How do I extract these few MB from the 10GB file ?

Answer 1

Basically, beacause you know how large your single sample is (32 bit = 4 B), and since you know where your signal of interest starts and how long it lasts, this is just a matter of

• Converting that position and length knowledge to sample numbers (if you know it in time rather than samples; but it's easy; $n_{sample}= t_{sample}\cdot f_{sample}$)
• Multiplying that with the sample size
• Cutting the file at the appropriate places

If you're on a unixoid (or have dd for windows), then dd can do that for you, with the bs (blocksize), with skip you can skip the start of the input, and with count= you can define how much you want. So, something like, if you want the 1 million samples, starting at sample 12345678:

 dd if=/your/large/sample/file of=/path/to/output/file bs=4 skip=12345677 count=1000000

Of course, there's other ways, too. If you like Python, you could use numpy to open that file using numpy.memmap, which acts as if you could directly deal with the data on disk as if it was in RAM; that alleviates the need to have 10 GB of spare RAM just to skip the first 9 GB. memmap takes an offset argument and hence doesn't require seeking through the unintersting beginning, and shape can be set to the length you're interested in:

import numpy as np
arr = np.memmap("/path/to/input", dtype=np.float32, offset=12345678, shape(1,1000000))
# maybe you'd even want to do some signal processing here?
arr.tofile("/path/to/output") 

Of course, within GNU Radio:

Open the file with file_source, skip the beginning with the skip_head block, keep only what you want with the head block.