Image segmentation

ANTs computes high-dimensional mappings to capture the statistics of brain structure and function.

Image Registration

Diffeomorphisms: SyN, Independent Evaluation: Klein, Murphy, Template Construction (2004)(2010), Similarity Metrics, Multivariate registration, Multiple modality analysis and statistical bias

Image Segmentation

Atropos Multivar-EM Segmentation (link), Multi-atlas methods (link) and JLF, Bias Correction (link), DiReCT cortical thickness (link), DiReCT in chimpanzees

vmtk is a collection of libraries and tools for 3D reconstruction, geometric analysis, mesh generation and surface data analysis for image-based modeling of blood vessels.

vmtk is composed of

• C++ classes (VTK and ITK -based algorithms)
• Python classes (high-level functionality - each class is a script)
• PypeS - Python pipeable scripts, a framework which enables vmtk scripts to interact with each other

SliceMap

Whole brain tissue slices are commonly used in neurobiological research for analyzing pathological features in an anatomically defined manner. However, since many pathologies are expressed in specific regions of the brain, it is necessary to have an annotation of the regions in the brain slices. Such an annotation can be done by manual delineation, as done most often, or by an automated region annotation tool.

SliceMap is a FIJI/ImageJ plugin for automated brain region annotation of fluorescent brain slices. The plugin uses a reference library of pre-annotated brain slices (the brain region templates) to annotate brain regions of unknown samples. To perform the region annotation, SliceMap registers the reference slices to the sample slice (using elastic registration plugin BUnwarpJ) and uses the resulting image transformations to morph the template regions towards the anatomical brain regions of the sample. The resulting brain regions are saved as FIJI/ImageJ ROI’s (Regions Of Interest) as a single zip-file for each sample slice.

More information can also be found in "SliceMap: an algorithm for automated brain region annotation", Michaël Barbier, Astrid Bottelbergs, Rony Nuydens, Andreas Ebneth, Winnok H De Vos, Bioinformatics, btx658, https://doi.org/10.1093/bioinformatics/btx658

SuRVoS: Super-Region Volume Segmentation workbench

A volume is first partitioned into Super-Regions (superpixels or supervoxels) and then interactively segmented by the user providing training annotations. SuRVoS can then learn from and extend the annotations to the whole volume.

This project was designed for vectorize and analyze the  blood vessels in the mouse brain.

This plugin requires the definition of seed point detection settings by the user (Semi-automated).

We have developed a novel approach, named DF-Tracing, to tackle this challenge. This method first extracts the neurite signal (foreground) from a noisy image by using anisotropic filtering and automated thresholding. Then, DF-Tracing executes a coupled distance-field (DF) algorithm on the extracted foreground neurite signal and reconstructs the neuron morphology automatically. Two distance-transform based “force” fields are used: one for “pressure”, which is the distance transform field of foreground pixels (voxels) to the background, and another for “thrust”, which is the distance transform field of the foreground pixels to an automatically determined seed point. The coupling of these two force fields can“push” a “rolling ball” quickly along the skeleton of a neuron, reconstructing the 3D cell morphology.

EBImage provides general purpose functionality for image processing and analysis. In the context of (high-throughput) microscopy-based cellular assays, EBImage offers tools to segment cells and extract quantitative cellular descriptors. This allows the automation of such tasks using the R programming language and facilitates the use of other tools in the R environment for signal processing, statistical modeling, machine learning and visualization with image data.

EBImage is available through the Bioconductor software project (www.bioconductor.org). Strengths Lightweight Suitable for automated, scripted analyses All functions are documented with examples Modular links to R and Bioconductor software, notably imageHTS and cellHTS2 Community support via the Bioconductor mailing list Reproducible (image) analysis using the Sweave report-writing system

Workflow of data-analysis strategies for image-based cell profiling.

1. Image analysis (image correction, image segmentation, feature extraction)
2. Image quality control
3. Preprocessing extracted features
4. Dimensionality reduction
5. Single-cell data aggregation
6. Measuring profile similarity
7. Assay quality assessment
8. Downstream analysis

Examples of publications where the workflow has been used at least partly:

• Image-based multivariate profiling of drug responses from single cells
• Phenotypic profiling of the human genome by time-lapse microscopy reveals cell division genes
• Mapping genetic interactions in human cancer cells with RNAi and multiparametric phenotyping
• Cell shape and the microenvironment regulate nuclear translocation of NF‐κB in breast epithelial and tumor cells
• Genome-wide RNAi Screening Identifies Protein Modules Required for 40S Subunit Synthesis in Human Cells
• Cell Painting, a high-content image-based assay for morphological profiling using multiplexed fluorescent dyes
• Systematic morphological profiling of human gene and allele function via Cell Painting

Microscopy Image Browser (MIB) is a high-performance Matlab-based software package for advanced image processing, segmentation and visualization of multi-dimensional (2D-4D) light and electron microscopy datasets.

MIB is a freely available, user-friendly software for effective image processing of multidimensional datasets that improves and facilitates the full utilization of acquired data and enables quantitative analysis of morphological features. Its open-source environment enables fine tuning and possibility of adding new plug-ins to customize the program for specific needs of any research project.

Spot detector detects and counts spots, based on wavelet transform.

- Detects spots in noisy images 2D/3D.
- Depending on objective, spots can be nuclei, nucleus or cell
- Versatile input: sequence or batch of file.
- Detects spot in specific band/channel.
- Multi band labeling: automaticaly creates ROIs from one band and count in the same or an other band.
- Filters detection by size.
- Sort detection by ROIs
- Output data in XLS Excel files: number of detection by ROIs, and each detection location and size.
- Outputs withness image with ROIs and detection painted on it.
- Outputs binary detection image.
- Displays detections
- Displays tags

This is the "prediction step" of the Pyxit segmentation model builder. It is a learnable segmentation algorithm based on ground-truth images and segmentation mask. It learns a multiple output pixel classification algorithm. It downloads from Cytomine-Core annotation images+alphamasks from project(s), build a segmentation (pixel classifier) model which is saved locally. Typical application: tumor detection in tissues in histology slides. 

This is a learnable segmentation algorithm based on ground-truth images and segmentation mask. It learns a multiple output pixel classification algorithm. It downloads from Cytomine-Core annotation images+alphamasks from project(s), build a segmentation (pixel classifier) model which is saved locally. Typical application: tumor detection in tissues in histology slides. It is based on "Fast Multi-Class Image Annotation with Random Subwindows and Multiple Output Randomized Trees" http://orbi.ulg.ac.be/handle/2268/12205 and was used in "A hybrid human-computer approach for large-scale image-based measurements using web services and machine learning" http://orbi.ulg.ac.be/handle/2268/162084?locale=en

This is the source code and data page of a distributed parallel algorithm 2683 for segmentation of large fluorescence microscopy images.

Image-processing algorithms developed at the MOSAIC Group for fluorescence microscopy. Tools included:

• 2D/3D single-particle tracking tool which can be used to track bright spots in 2D/3D movies over time.
• Optimal filament segmentation of 2D images. 
• Curvature filters for image filtering, denoising, and restoration. 
• Image naturalization for image enhancement based on gradient statistics of natural-scence images. 
• Tool for automatically send and distribute jobs on clusters and get back the results.
• Multi-region image segmentation of 2D and 3D images without needing to know the number of regions beforehand. 
• Squassh for globally optimal segmentation of piecewise constant regions in 2D and 3D images and for object-based co-localization analysis. 
• Tool for inferring spatial interactions between patterns of objects in images or between coordinates read from a file.
• Tool for robust, histogram-based background subtraction well suited to correct for inhomogeneous illumination artifacts.
• A tool to estimate the Point-Spread Function of the microscopy out of 2D fluorescence images.
• A tool to measure the 3D Point-Spread Function of a confocal microscope from an image stack.
• Addition of synthetic Poisson-distributed noise to an image in order to simulate shot noise of various signal-to-noise ratios. 
• Convolution of an image with a Bessel function in order to simulate imaging with a microscope. 
• A utility to detect bright spots in images and estimate their center. 
• A utility to create manual segmentations to be used as ground truth to test and benchmark automatic segmentation algorithms.
• A tool for replacing one color in an image with another color.

Free-D is a three-dimensional (3D) reconstruction and modeling software. It allows to generate, process and analyze 3D point and surface models from stacks of 2D images. Free-D is an integrated software, offering in a single graphical user interface all the functionalities required for 3D modeling. It runs on Linux, Windows, and MacOS. Free-D is developed by the Modeling and Digital Imaging team of the Institut Jean-Pierre Bourgin, INRA Versailles, France.

Free-D (http://free-d.versailles.inra.fr/) is a 3D reconstruction and modeling software. It is multiplatform, free (but not open source) tool for academic research and teaching.

Here is how to proceed, using Free-D:

1. Segmentation:

* load (a collection of) individual 3d stacks

* (optional for serial sections) perform a 2D registration to align image slices

* segment/reconstruct 3D contours using snakes

* segment 3D spots

2. Construct average cell:

* normalize the contours to compute a average cell, by registering/warping 3D contours/surfaces

3. Quantification:

* project each individual cell to the average one

* build density maps to analyze (cartography)

A few notes for current software version (till 10/2016):

* input file format: tiff (not able to import bioformats)

* currently results are saved in customized format, but there is an exportor to convert this format into fiji readable one

* import already generated contours is on the software's TODO list

It is a tool to visualize and annotate volume image data of electron microscopy. Users can annotate objects (e.g. neurons) and skeleton structures. It provides the ability to overlaying the image data with user annotations, representing the spatial structure and the connectivity of labeled objects, and displaying a three dimensional model of it. It can be extended by plugins written in python. A similar, web-based implementation is being developed at webknossos.info. Example datasets are also available.

MorphoLibJ is a library of plugin for ImageJ with functionalities for image processing such as filtering, reconstructing, segmenting, etc... Tools are based on Mathematical morphology with more rigorous mathematical approach than in the standard tools of ImageJ in particular for surface (or perimeter) measurements which are usually based on voxel counting.  

http://imagej.net/MorphoLibJ#Measurements

Among the features:

Morphological operations :  Dilation, Erosion, Opening,  Closing , Top hat (white and black), Morphological gradient (aka Beucher Gradient), Morphological Laplacian, Morphological reconstruction, Maxima/Minima , Extended Maxima/Minima -Watershed (classic or controlled) -Image overlay -Image labelling -Geodesic diameter -Region Adjacency Graph -Granulometry curves, morphological image analysis.

When trying to isolate objects, one strategy might be to use regular morphological operations (opening/closing) to remove small objects that are not of interest. In case small objects are made of a large number of pixels, this operation might impair the remaining objects' contours. An alternative strategy might be to use morphological reconstruction. In short, seed is placed on the image, on objects, then conditional dilation is performed from those seeds.

Here is how to proceed, using MorphoLibJ:

1. Open an image
2. Use the multi-point selection tool and place seeds on objects of interest
3. Create a new image of same size, black background
4. Transfer the selection to the new image (Edit/Selection/Restore selection)
5. Draw (make sure you're using white foreground) the multiple point selection
6. Launch the Morphological reconstruction plugin: Plugins > MorphoLibJ > Morphological reconstruction

ABSnake can segment complex structures in 2D images as well as 3D or temporal images. It uses a new active contour model based on a geometrical approach for correctly following invaginated structures.

MosaicIA is a tool to analyze the spatial distribution of objects in images. It estimates from an observed particle or object distribution what hypothetical interaction between the objects is most likely to have created this distribution.

In this case study, MATLAB, the Image Processing and Signal Processing toolboxes were used to determine the green intensities from a small portion of a microarray image containing 4,800 spots. A 10x10 pattern of spots was detected by averaging rows and columns to produce horizontal and vertical profiles. Periodicity was determined automatically by autocorrelation and used to form an optimal length filter for morphological background removal. A rectangular grid of bounding boxes was defined. Each spot was individually addressed and segmented by thresholding to form a mask. The mask was used to isolate each spot from surrounding background. Individual spot intensity was determined by integrating pixel intensities. Finally, integrated intensities were tabulated and saved to a data file for subsequent statistical analysis to determine which genes matter most.

Quote

Perfect for the beginner, this demo illustrates simple object detection (segmentation, feature extraction), measurement, and filtering. Requires the Image Processing Toolbox (IPT) because it demonstrates some functions supplied by that toolbox, plus it uses the "coins" demo image supplied with that toolbox. If you have the IPT (you can check by typing ver on the command line), you should be able to run this demo code simply by copying and pasting this code into a new editor window, and then clicking the green "run" triangle on the toolbar. First finds all the objects, then filters results to pick out objects of certain sizes. The basic concepts of thresholding, labeling, and regionprops are demonstrated with a simple example.

It's a good tutorial for those users new to MATLAB's image processing capabilities to learn on, before they go on to more sophisticated algorithms.

A workflow template to analyze subcellular structures in fluorescence 2D/3D microscopy images based on a Fiji plugin **Squassh** is described in Rizek et al (2014).

The workflow employs detecting, segmenting, and quantifying subcellular structures. For segmentation, it accounts for the microscope optics and for uneven image background. Further analyses include both colocalization and shape analyses. However, it does not work directly for time-lapse data. A brief summary note can be found here.

An example macro introduced in the documentation page of the ImageJ plugin Trainable Weka Segmentation (in Fiji, it's bundled). A segmentation protocol based on machine learning. Full macro is available in the "Download" Link. 

This plugin can be trained to learn from the user input and perform later the same task in unknown (test) data. Weka: it makes use of all the powerful tools and classifiers from the latest version of Weka. Segmentation: it provides a labeled result based on the training of a chosen classifier. Trainable Weka Segmentation Complete macro example is at the end of the page.

This article Baslat et al. presents a method to compute Lymphatic Vessel Density on an image of the whole slide (a workflow documented as text).

Vessels are obtained with a Maximum Entropy Thresholding applied on the excess Red channel (2 times the red values minus blue+green value). Stroma tissue is obtained with a Moment Preserving Thresholding on the blue channel.

CellX is an open-source software package of workflow template for cell segmentation, intensity quantification, and cell tracking on a variety of microscopy images with distinguishable cell boundary.

Installation and step-by-step usage details are described in Mayer et al (2013). 

After users provide a few annotations of cell sizes and cell boundary profiles, it tries to match boundary profile pattern on cells thus provide segmentation and further tracking. It works the best on cells without extreme shapes and with a rather homogeneous boundary pattern. It may not work well on images with cells of sizes only a few pixels. Its output comprises control images for visual validation, text files for post-processing statistics, and MATLAB objects for advanced subsequent analysis.

This macro recognizes wells in a picture from a multi-well plate (it works also on a picture of a single well). It is used to segment a picture to determine the number of "Colony Forming Units" in each individual well of a plate.

The steps are the following:

1. Makes a 8-bit B&W picture, inverts it (=> borders will look white instead of black), resizes it (optional, this is to speed-up convolution thereafter) and find edges.
2. Convolves the obtained picture with a kernel corresponding to a thick white circle of the size of the wells. The resulting image has big "blobs" or "particles" corresponding roughly to the centers of the well.
3. The image is thresholded to remove particles not corresponding to strong hits and "Analyze particle" is run.
4. The measured parameter is the center of mass of the particles which gives the center of the well. These are saved in an array.
5. Circles are drawn and added to the ROI manager. The centers of the circles are the identified centers of mass of the particles and their radius is the expected radius of the wells in the original image.

This method allows to compute a threshold that preserves the moments of an image. In ImageJ/Fiji, you can access it in Image->Ajust->Threshold and choose Moments in the list. In Aphelion, the tool is in Segmentation->Threshold->AphImgMomentThreshold The original paper is 2449

SlideToolkit is a collection of command-line tools to assist with the automated histology analysis of whole-slide images. The publication linked in the "reference" details the actual workflow. 

This includes tools to organize the data, perform tiling and subsequent batch processing of the generated tiles in a cell profiler pipeline. All the tools are designed to run on a single PC or on a HPC system. The scripts in the toolkit are on github under MIT licence.

This workflow classifies, or segments, the pixels of an image given user annotations. It is especially suited if the objects of interests are visually (brightness, color, texture) distinct from their surrounding. Users can iteratively select pixel features and provide pixel annotations through a live visualization of selected feature values and current prediction responses. Upon users' satisfaction, the workflow then predicts the remaining unprocessed image(s) regions or new images (as batch processing). Users can export (as images of various formats): selected features, annotations, predicted classification probability, simple segmentation, etc. This workflow is often served as one of the first step options for other workflows offered by ilastik, such as object classification, automatic tracking.

Requires Matlab Runtime Environment or Matlab. Source code (m-files) are downloaded. Software availability: AVeMap was developed under MATLAB (MathWorks). It is available as an executable, multiplatform program, together with source codes and documentation here, and the source code is also available as Supplementary Software. For practical reasons, this executable version, which does not require MATLAB, runs on a single processor. For users who want to customize the software and/or need the power of parallel computing, an installation of MATLAB with its 'parallel' and 'image processing' toolboxes is needed. Note that, even with the executable version, the velocity fields are stored for further analysis. The add-on AVeMap+ uses these AVeMap-computed velocity fields to generate heat map tables. It is available with the same link.

In this example, cells are grown as a tissue monolayer. Rather than identifying individual cells, this pipeline quantifies the area occupied by the tissue sample.

Download package also contains example images. 

This workflow classifies objects based on object-level features (e.g. intensity based, morphology based, etc) and user annotations. It needs segmentation images besides the raw image data. Segmentation images can be obtained from ilastik pixel classification, or binary segmentation images from other tools. Within the object classification, one can prefilter objects through thresholds (on pixel probability image) or object sizes (on segmentation image). Outputs are predicted classification label images. Selected features can also be exported. Advanced users also have possibilities to add customized (object) features for classification in a simple plugin fashion through python scripts.

This workflow is used to track multiple (appear/disappear, dividing and merging) objects in presumably big 2D+t or 3D+t datasets. It is best suitable for roundish objects or spots. Tracking is done through segmentation, which can be obtained from ilastik pixel classification, or imported from other tools. Users should provide a few object level labels, and the software predicts results on the rest of the image or new images with similar image characteristics. As a result, all objects get assigned random IDs at the first frame of the image sequence and all descendants in the same track (also children objects such as daughter cells) inherit this ID.

The original paper describes a method to analyze mitochondrial morphology in 2D and 3D.

Task

Quantify the length of microtubules (MT) and the MT average density per cell.

Workflow descriptions

Simple two step workflow, allowing visual & manual correction of microtubule between the 2 steps. Batch measurement of microtubule lengths for multiple images is achieved by segmenting the MTs and then their skeletonizations. The number of pixels in the microtubule is proportional to their length, so the length can be estimated.

Script

Workflow is written as an ImageJ macro (Fiji) with following steps:

1. The enhancement of tubular structure by computing eigenvalues of the hessian matrix on a Gaussian filtered version of the image ( sigma 1 pixel), as implemented in the tubeness plugin.

2. The tubules were then thresholded , and structures containing less than 3 pixels were discarded.

3. If needed, a visual check and correction of segmented microtubule is then performed.

4. After correction, segmented MTs were then reduced to a 1-pixel thick line using the skeletonize plugin of Fiji. The length of the skeletonized microtubules was then directly proportional to their length.

5. Data were grouped by condition and converted back to micrometers units under Matlab for the statistical tests.

Pitfalls

Commented but not that general without editing some fields in the macros.

Sample Data

Sample data and workflow (see above URL) can be accessed by - login: biii - password Biii!

Misc

3D version also available here. Use of components Skeletonize and Tubeness Filter

Tracking of focal adhesions includes a number of challenges:

1. Detection of focal adhesion regions in areas of highly variable background
2. Separation of "clumped" adhesions in different objects.
3. Dynamics: Focal adhesions dynamically, grow, shrink, change their shape, they can fuse with neighboring adhesions or one adhesion can be split into multiple children.

Würflinger et al (2011) describe how to detect focal adhesion objects and how to track them over time. Interestingly, tracking results are fed back to segmentation to improve separation of clumped adhesions.

The authors implemented the workflow in Matlab, but do not provide a ready-to-use script.

This macro segments blood vessels in a 3D stack. It is suited for well-contrasted images (low background) and works better if the width of the vessels of interest is reasonably uniform.

Sample image: 1

sample image: 2

This macro performs measurements of average and standard deviation intensity inside wells of a protein microarray (the number of wells is limited to 250, the image should be cropped for larger arrays). The macro requires the "ImageJ plugins toolkit". To ensure compatibility with Fiji you should download the version 1.6.1. The installation instructions can be found here, it only consists in un-compressing the .jar file from the previous archive to Fiji plugins folder.

sample image: link

Quote:

This pipeline shows how to identify smaller objects (foci) within larger objects (nuclei) and how to use the Relate module to establish a relationship between the two as well as perform per-object aggregate measurements (such as number of foci per nucleus). Sample images are included in the download package.

Simple workflow description for ImageJ, step-by-step description for delineating focal adhesions, count and characterize their positions.  

Measurement of dynamics is not involved.

This macro is meant to segment the cells of a multicellular tissue. It is written for images showing highly contrasted and uniformly stained cell membranes. The geometry of the cells and their organization is automatically extracted and exported to an ImageJ results table. This includes: Cell area, major, minor fitted ellipse radii + major axis orientation and number of neighbors of the cells. Manual correction of the automatic segmentation is supported (merge split cells, split merged cells).

Sample image data is available in the documentation page. 

Simple macro to separates blobs.

• Load the ImageJ sample image "Blobs"
• Run the plugin Morphological Segmentation
• Display the overlaid

This macro depends on "Morphological Segmentation" component of the MorphoLibJ library, which should be installed via update sites.

This macro was designed to measure the size of the scratch wound in a wound scratch assay. It uses an edge-detection and thresholding technique.

It will batch process all images in a directory. Images captured by time-lapse should be compiled into stacks using a tool similar to "Metamorph nd & ROI files importer (nd stack builder)" by Fabrice P. Cordelières. Images to be analyzed should be placed in one directory (Source Directory). A second directory should be created to save results files and images (Destination Directory). Setting correct Lower and Upper thresholds is important to obtain a good result. Two macros are available, one using edge detection, the second one using background subtraction.

ITK-SNAP is a software application used to segment structures in 3D medical images. It can also be used as a 3D annotation tool for deep learning. It is based on ITK, VTK libraries.

**Collaborative Annotation Toolkit for Massive Amounts of Image Data** CATMAID is a Collaborative Annotation Toolkit for Massive Amounts of Image Data. It is designed to navigate, share and collaboratively annotate massive image data sets of biological specimens. The interface is inspired by GoogleMaps, with which it shares basic navigation concepts, enhanced to allow the exploration of 3D biological image data acquired by optical or physical sectioning microscopy techniques. The interface enables seamless sharing of regions of interest through bookmarks and synchronized navigation through multiple registered data sets. With massive biological image data sets it is unrealistic to create a sustainable centralized repository. A unique feature of CATMAID is its partially decentralized architecture where the presented image data can reside on any Internet accessible server and yet can be easily cross-referenced in the central database. In this way no image data are duplicated and the data producers retain full control over their images. CATMAID is intended to serve as data sharing platform for biologists using high-resolution imaging techniques to probe large specimens. Any high-throughput, high-content imaging project such as gene expression pattern screens would benefit from the interface for data sharing and annotation.

Acquiarium is for carrying out the common pipeline of many spatial cell studies using fluorescence microscopy. It addresses image capture, raw image correction, image segmentation, quantification of segmented objects and their spatial arrangement, volume rendering, and statistical evaluation. It is focused on quantification of spatial properties of many objects and their mutual spatial relations in a collection of many 3D images. It can be used for analysis of a collection of 2D images or time lapse series of 2D or 3D images as well. It has a modular design and is extensible via plug-ins. It is a stand-alone, easy to install application written in C++ language. The GUI is written using cross-platform wxWidgets library.

ITK is an open-source, cross-platform system that provides developers with an extensive suite of software tools for image analysis.

Developed through extreme programming methodologies, ITK employs leading-edge algorithms for registering and segmenting multidimensional data. It is widely used and contributed in the medical imaging field.

Strengths

Highly optimized C++, well commented Consistently updated (new) algorithms many tools and softwares are built upon it connected with VTK Insight Journal (open code and sample data) Extensive list of examples & tutorials

Limitations

yet detached from the bioimage analysis world hard to use for end users without development skills