Introduction
When data logging is enabled during line scanning, the data is written to disk in multiple files to facilitate efficient processing. This helps enable higher sample rates due to low processing overhead. Each line scanning GRAB session will create 3 files:
- [File name stem][File counter].meta.txt (Ex: filename_00001.meta.txt): Plain text file containing metadata describing the ScanImage parameters and stimulus pattern used for the scan
- [File name stem][File counter].pmt.dat (Ex: filename_00001.pmt.dat): Binary data file containing the collected fluorescence data
- [File name stem][File counter].scnnr.dat (Ex: filename_00001.scnnr.dat): Binary data file containing the collected scanner position feedback data (only created if position monitoring is enabled)
ScanImage includes a utility function to read these files and load the data into a convenient structure.
Using the ScanImage Utility
The ScanImage utility to load a data file can be called with the following signature:
[header, pmtData, scannerPosData, roiGroup] = scanimage.util.readLineScanDataFiles(fileName)
The meta data and data files should exist in the same directory and fileName should be entered without the extension (ex: "filename_00001"). The return values are the following:
- header: A hierarchical data structure containing the ScanImage parameters used when the data was collected as well as useful statistics about the total number of samples and frames collected and sample rate information. Reference the ScanImage API for more information about the contents of data log headers.
pmtData: A matrix of the collected fluorescence data organized by channel and frame. The size of the matrix is NxCxF where N is the number of samples per scan cycle, C is the number of channels, and F is the number of frames (cycles) collected. Index this array appropriately to extract the data you are interested in. To plot data collected during the 100th frame on channel 2, for instance, you could do the following:
plot(squeeze(pmtData(:,2,100))); % squeeze is needed to remove the first dimension since it becomes singleton
To plot data at point 600 from channel 1 over every frame (ie a plot of how fluorescence at a single point is changing over time) you could do the following:
plot(squeeze(pmtData(600,1,:))); % squeeze is needed to remove the third dimension since it becomes singleton
scannerPosData: If scanner position monitoring was on, a structure with a fields "G" and "Z" containing matrices for recorded scanner position data for scan mirrors and the Z actuator respectively. The size of the G matrix is Nx2xF where N is the number of data points per cycle and F is the number of frames collected. The columns correspond to the X and Y mirrors. The size of the Z matrix is Nx1xF where N is the number of data points per cycle and F is the number of frames collected. To plot the path of the X scan mirror collected during the 100th frame, for instance, you could do the following:
plot(squeeze(scannerPosData.G(:,1,100))); % squeeze is needed to remove the first dimension since it becomes singleton
To plot the position of the Y scan mirror at point 600 from over every frame (ie a plot of the repeat-ability of scan mirror position at that point in the path) you could do the following:
plot(squeeze(scannerPosData.G(600,2,:))); % squeeze is needed to remove the third dimension since it becomes singleton
- roiGroup: A copy of the stimulation pattern that was used to collect the data. Reference the ScanImage API for more information about how to work with ROI Group objects.
Manually Decoding Data Files
If it is desired to write your own code to load the files (for instance if you need to perform analysis outside of MATLAB) the following provides information about how to read the files.
Metadata File: (Ex: filename_00001.meta.txt): This is a plain text file. If the option is enabled to store metadata in JSON format, there will be two JSON structures. The first will be a hierarchical data structure containing the ScanImage parameters used when the data was collected. The second structure will be a data structure representing the stimulus pattern that defines the scan path used to collect the data. If the JSON option is not enabled, the ScanImage parameters will be written with "dot" syntax to represent the data sctructure, ie:
SI.hScan2D.sampleRate = 1.2e+08
In this case, the stimulation pattern info following the ScanImage parameter data will still be JSON. You can detect if the ScanImage data is stored as JSON by checking if the first character of the file it "{".
Fluorescence Data File: (Ex: filename_00001.pmt.dat): This file contains raw binary data of samples written as signed 16-bit integers. The order that samples are written is:
[Channel 1 Sample 1, Channel 2 Sample 1, ..., Channel N Sample 1, Channel 1 Sample 2, Channel 2 Sample 2, ..., Channel N Sample 2, ..., Channel 1 Sample M, Channel 2 Sample M, ... Channel N Sample M]
where N is the number of channels and M is the number of samples per channel. The number of channels can be found in the header by counting the number of elements in SI.hChannels.channelSave. The number of samples per frame can be found at SI.hScan2D.lineScanSamplesPerFrame. The acquisition sample rate can be found at SI.hScan2D.sampleRate.
Scanner Position Feedback Data File: (Ex: filename_00001.scnnr.dat): This file contains raw binary data of samples written in single precision. The order that samples are written is:
[Channel 1 Sample 1, Channel 2 Sample 1, ..., Channel N Sample 1, Channel 1 Sample 2, Channel 2 Sample 2, ..., Channel N Sample 2, ..., Channel 1 Sample M, Channel 2 Sample M, ... Channel N Sample M]
where N is the number of channels and M is the number of samples per channel. The number of channels can be found in the header at SI.hScan2D.lineScanNumFdbkChannels. For a 2D scan there will be two channels: X and Y. For a 3D scan there will be three channels: X, Y, and Z. The number of samples per frame can be found at SI.hScan2D.lineScanFdbkSamplesPerFrame. The feedback sample rate can be found at SI.hScan2D.sampleRateFdbk.
