
File Structure¶
Fly-QMA uses a standardized file structure, and will automatically adhere to this format by creating and updating various subdirectories and files as needed. The file structure is hierarchically organized into three levels:
EXPERIMENT: One or more tissue samples imaged under the same conditions.
STACK: All images of a particular tissue sample, such as an individual z-stack.
LAYER: All analysis relevant to a single 2-D image, such as an individual layer.
Experiments¶
Microscopy data should be arranged into a collection of STACK directories that reside within an EXPERIMENT directory unique to a particular set of experimental conditions:
EXPERIMENT
│
├── STACK 0 # First STACK directory
├── STACK 1
└── ... STACK N # Nth STACK directory
Image Stacks¶
Each STACK directory contains various components pertinent to all images within the image z-stack. These may include:
The original
.tif
image file depicting a z-stack of an imaginal disc. Images may be regularly-spaced 3D z-stacks or irregularly-spaced 3D collections of one or more layers. If a 2D image is provided, Fly-QMA will assume the z-stack only contains a single layer.A
metadata.json
file containing all imaging metadata, e.g. number of layers, number of fluorescence channels, image bit depth, etc.An
annotation
subdirectory containing all of the model components used to annotate a particular image stack.A
layers
subdirectory containing all of the lower level LAYER directories. Layers are sequentially numbered, beginning with zero.
EXPERIMENT
│
├── STACK 0
│ ├── image.tif
│ ├── metadata.json
│ ├── annotation
│ └── layers
│ ├── 0 # first LAYER directory
│ ├── 1
│ └── ... N # Nth LAYER directory
├── STACK 1
│
└── ... STACK N
Layers¶
Each LAYER directory contains all components pertinent to an individual 2D layer within the z-stack. These may include:
A
metadata.json
file containing all layer metadata, such as particular parameter values used.A
selection
subdirectory containing aselection.npy
ROI mask. This mask is a binary 2D numpy array in which each element denotes whether a given pixel is within the ROI. Theselection
directory also includes amd.json
file used whose contents are used to indicate whether or not the layer is included within subsequent analyses.A
correction
subdirectory containing a parameterized model for performing bleedthrough correction. Thedata.json
file contains the model parameterization, whilefit.png
depicts the model fit andcorrection.png
shows the resultant correction.A
segmentation
subdirectory containing alabels.npy
segmentation mask. This mask is a 2D numpy array of integers in which each element represents a single pixel within the image. The integer value denotes the segment assigned to each pixel, where zero-valued pixels comprise the background. Thesegmentation
directory may also include an image of the resultant segmentation, stored assegmentation.ong
, but this file is not required.A
measurements
subdirectory containing two serialized Pandas dataframes. The filemeasurements.hdf
contains the raw measured pixel intensities for all detected cells or nuclei, whileprocessed.hdf
contains a cached version of the measured data after all analyses (e.g. bleedthrough correction, annotation, etc.) have been applied. The former is used to preserve the original measurements, while the latter is used to cache the results of previous analysis so they may be rapidly retrieved at any time.
EXPERIMENT
│
├── STACK 0
│ ├── image.tif
│ ├── metadata.json
│ ├── annotation
│ └── layers
│ ├── 0
│ │ ├── metadata.json
│ │ │
│ │ ├── selection # ROI selection subdirectory
│ │ │ ├── md.json
│ │ │ └── selection.npy # vertices defining ROI
│ │ │
│ │ ├── correction # bleedthrough correction subdirectory
│ │ │ ├── data.json
│ │ │ ├── fit.png
│ │ │ └── correction.png
│ │ │
│ │ ├── segmentation
│ │ │ ├── labels.npy # segmentation mask (np.ndarray[int])
│ │ │ └── segmentation.png # layer image overlayed with segment contours (optional)
│ │ │
│ │ └── measurements
│ │ ├── measurements.hdf # raw expression measurements
│ │ └── processed.hdf # processed expression measurements
│ │
│ ├── 1
│ └── ... N
├── STACK 1
└── ... STACK N
Note
It is possible to integrate external analyses into the Fly-QMA workflow by manually adding them in accordance with the standardized file structure. For instance, users may import their own ROI or segmentation masks by adding them to the appropriate subdirectories. However, Fly-QMA also provides a handful of import methods designed explicitly for this purpose. See the integration section for additional details.
Annotation¶
In Fly-QMA, annotation entails training a model to identify distinct levels of clonal marker fluorescence, then applying the model within the spatial context of a given image. While annotation is always applied at the LAYER level, Fly-QMA supports training the annotation model on each LAYER or on the entire STACK. The annotation
subdirectory resides at the level used to train the model. Its contents are detailed below. If a model selection procedure is used, all of the trained models are also cached within a models
subdirectory.
EXPERIMENT
│
├── STACK 0
│ ├── image.tif
│ ├── metadata.json
│ ├── layers
│ └── annotation # annotation subdirectory
│ │
│ ├── annotation.json # annotation parameters
│ │
│ ├── classifier # selected model directory
│ │ ├── parameters.json # selected model parameters
│ │ ├── model.pkl # pickled mixture model
│ │ └── values.npy # data used to fit mixture model
│ │
│ └── models # model selection directory
│ ├── parameters.json # model selection parameters
│ ├── values.npy # data used for model selection
│ ├── classifier_0
│ ├── classifier_1
│ └── ... classifier_M # Mth mixture model directory
├── STACK 1
└── ... STACK N