Data Storage#

During the analysis we make use of hdf5 files to store data. These are faster to access than csv or similar text files, while being much more space efficient. Importantly, they allow for lossless storage of data while retaining the correct data format (dates or complex numbers are particularly troublesome in csv) and for storage of multiple tables that allows storage of secondary data in a much cleaner manner.

Bindings exist for most major programming languages or with the standalone HDFView software.

fourier.h5#

These intermediary files contain the locations of the objects of interest within the microscope files and the Fourier components of their boundaries. The files are created by the image processing stage of FlickerPrint, with one file produced per microscope file.

This file is split into two components:

fourier#

Primarily contains a pandas.DataFrame (table) with the headers:

im_path:

str

The path of the microscope image used for the analysis

frame:

int

The frame number for which we record the data

granule_id:

int

Each granule is assigned an ID as it is tracked across the frames

order:

int

Fluctuation order, q

magnitude:

float_complex

Complex magnitude of the given mode q for this granule

x, y:

float, float

Position of the granule centre in pixel coordinates

bbox_left, bbox_bottom, bbox_right, bbox_top:

float, float, float, float

Positions of the corners of the box bounding the granule

mean_radius:

float

Radius of the granule

valid:

bool

If False then a discontinuity has been detected in the boundary drawing for this frame

major_axis:

float

Fit an ellipse to the granule and measure the major axis of the in pixels

eccentricty:

float

Fit an ellipse to the granule and measure the eccentricty

timestamp:

str

The date and time that this frame was taken, if available. Otherwise zero

Attributes#

These data are attached to the fourier table as attributes to provide further information:

num_frames:

int

The total number of frames in the microscope image file

input_path:

str

The path of the file used in the analysis

pixel_size:

float

The physical size on the sample of a pixel in the image (μm/pixel)

config:

str

A copy of the config file used to run the “process-image” step that produced this file

version:

str

The version number of the Granule Explorer code that produced this file

aggregate_fittings.h5#

The aggregate fittings file is the final output of the main FlickerPrint workflow. This file contains tables listing the granule properties and fitting information. One file is produced per experiment directory.

This is split into three components:

aggregate_data#

A pandas.DataFrame containing a list of granule properties, with the columns:

granule_id:

int

Each granule is assigned an ID during the image processing step, note that this is only unique when combined with the image_path.

sigma:

float

Best fit surface tension for the granule in units of N/m

sigma_err:

float

The estimated error of sigma in N/m

kappa_scale:

float

Best fit bending rigidity for the granule, measured in units of k B T

kappa_scale_err:

float

The estimated error of kappa in k B T

mean_radius:

float

Mean radius of the granule in μm

figure_path:

str

The name of the spectrum fitting figure produced by the analysis.

image_path:

str

The name of the file used to create the granule information in the image processing step

pass_rate:

float

The percentage of frames that have valid boundaries drawn image-processing, used to exclude the granule if too many frames fail

pass_count:

int

The number of frames in which this this granule appears and has a valid boundary drawn in image-processing. Used to exclude the granule if it only appears in a few frames

fitting_error:

float

The fitting error between the experimental fluctuation spectrum and the theoretical spectrum

durbin_watson:

float

The Durbin-Watson statistic for the spectrum (a measure of correlation between residuals of adjacent Fourier modes), used to determine whether a model is a good fit to the data. See here for details.

mean_intensity:

float

The mean intensity of pixels belonging to the granule (arb units)

x, y:

float , float

Position of the granule centre in pixel coordinates

bbox_left, bbox_bottom, bbox_right, bbox_top:

float, float, float, float

Positions of the corners of the box bounding the granule

q_2_mag:

float

The average magnitude of the second fluctuation mode. Gives an estimate of the circularity of the granule

experiment:

str

A string name of the experiment, used to combine seperate files from the same experiment, or compare different experiments

timestamp:

str

A the date and time that the granule first appears in the microscope file

fitting_diff:

float

The difference in fitting error between models which include interfacial tension only and both interfacial tension and bending rigidity

sigma_st:

float The interfacial tension as measured using the interfacial tension only model.

sigma_err_st:

float

The error on interfacial tension when using the interfacial tension only model.

above_res_threshold:

bool

A boolean indicating whether the granule has at least half of its fluctuation modes above the resolution threshold of the video.

Attributes#

The aggregate_data frame has the following attributes:

config:

str

A copy of the config file used to run the “spectrum-fitting” step.

version:

str

The version number of the Granule Explorer code that produced this file.

fourier_terms#

A second frame containing spectrum information for each object. This contains the following columns:

order:

int

The peturbation order, q.

mag_squ_mean:

float

The total perturbations from a circular granule: the mean of the absolute magnitude squared

mag_mean:

float_complex

The mean of the complex perturbations across the surface.

fixed_squ:

float

The squared magnitude of the static components.

fluct_squ:

float

The squared magnitude of the fluctuating components on top of the quasi-spherical static shape.

experimental_spectrum:

float

The spectrum used in fitting, it is typically fluct_squ unless set otherwise by the configuration file.

best-fit:

float

The best fit theoretical spectrum to the experimental_spectrum

granule_id:

int

Each granule is assigned an ID during the image processing step, note that this is only unique when combined with the figure_path.

figure_path:

str

Output figure path