Machine learning

An implementation of Belief Propagation for factor graphs, also known as the sum-product algorithm

This workflow predict landmark positions on images by using LC landmark detection models.

This workflow trains LC landmark detection models from a dataset of annotated images.

This workflow predict landmark positions on images by using MSET landmark detection models.

This workflow trains MSET landmark detection models from a dataset of annotated images.

PyTorch is an open-source machine learning library for Python, based on Torch, used for applications such as natural language processing.

Collection of several basic standard image segmentation methods focusing on medical imaging. In particular, the key block/applications are (un)supervised image segmentation using superpixels, object centre detection and region growing with a shape prior. Besides the open-source code, there is also a few sample images.

ImJoy is a plugin powered hybrid computing platform for deploying deep learning applications such as advanced image analysis tools.

ImJoy runs on mobile and desktop environment cross different operating systems, plugins can run in the browser, localhost, remote and cloud servers.

With ImJoy, delivering Deep Learning tools to the end users is simple and easy thanks to its flexible plugin system and sharable plugin URL. Developer can easily add rich and interactive web interfaces to existing Python code.

This workflow describes a semi-automatic image segmentation procedure for 3D reconstructions of the coronary arterial tree, after which how different morphometric features are automatically extracted, including vessel lumen diameter of the three main coronaries.

Labkit is an open-source tool to segment truly large image data using sparse training data. It has an intuitive and responsive user interface based on Big Data Viewer, allowing users to conveniently browse and annotate even terabyte sized image volumes.

Update site: Labkit

Scikit-learn (sklearn) is a python library used for machine learning. sklearn contains simple and efficient tools for data mining and data analysis. Modules and functions include those for classification, regression, clustering, dimensionality reduction, model selection and data preprocessing. Many people have contributed to sklearn (list of authors)

3-D density kernel estimation (DKE-3-D) method, utilises an ensemble of random decision trees for counting objects in 3D images. DKE-3-D avoids the problem of discrete object identification and segmentation, common to many existing 3-D counting techniques, and outperforms other methods when quantification of densely packed and heterogeneous objects is desired. 

Please also check out ImageJ and Fiji.

Biocat is a java based software that allows to perform image classification or segmentation using machine learning. Several algorithm for the classification are available.

ZEN and APEER – Open Ecosystem for integrated Machine-Learning Workflows

Open ecosystem for integrated machine-learning workflows to train and use machine-learning models for image processing and image analysis inside the ZEN software or on the APEER cloud-based platform

Highlights ZEN

• Simple User Interface for Labeling and Training
• Engineered Features Sets and Deep Feature Extraction + Random Forrest for Semantic Segmentation
• Object Classification workflows
• Probability Thresholds and Conditional Random Fields
• Import your own trained models as *.czann files (see: czmodel · PyPI)
• Import "AIModel Containes" from arivis AI for advanced Instance Segmentation
• Integration into ZEN Measurement Framework
• Support for Multi-dimensional Datasets and Tile Images
• open and standardized format to store trained models

ZEN Intellesis Segmentation

ZEN Intellesis - Pretrained Networks

Intellesis Object Classification

Highlights Aarivis AI

• Web-based tool to label datasets to train Deep Neural Networks
• Fully automated hyper-parameter tuning
• Export of trained models for semantic segmentation and AIModelContainer for Instance Segmentation

Annotation Tool

Chainer is a Python-based deep learning framework aiming at flexibility. It provides automatic differentiation APIs based on the define-by-run approach (a.k.a. dynamic computational graphs) as well as object-oriented high-level APIs to build and train neural networks. It also supports CUDA/cuDNN using CuPy for high performance training and inference. For more details of Chainer, see the documents and resources listed above and join the community in Forum, Slack, and Twitter.

Python is a programming language.

Python 2.7.0 was released on July 3rd, 2010.

Python 2.7 is scheduled to be the last major version in the 2.x series before it moves into an extended maintenance period. This release contains many of the features that were first released in Python 3.1.

 A bugfix release, 2.7.16, is currently available. Its use is recommended.

Quantitative Criterion Acquisition Network (QCA Net) performs instance segmentation of 3D fluorescence microscopic images. QCA Net consists of Nuclear Segmentation Network (NSN) that learned nuclear segmentation task and Nuclear Detection Network (NDN) that learned nuclear identification task. QCA Net performs instance segmentation of the time-series 3D fluorescence microscopic images at each time point, and the quantitative criteria for mouse development are extracted from the acquired time-series segmentation image. The detailed information on this program is described in our manuscript posted on bioRxiv.

Python is a programming language.

An ImageJ plugin for DEFCoN, the fluorescence spot counter based on fully convolutional neural networks

NiftyNet is a TensorFlow-based open-source convolutional neural networks (CNNs) platform for research in medical image analysis and image-guided therapy. NiftyNet’s modular structure is designed for sharing networks and pre-trained models. Using this modular structure you can:

• Get started with established pre-trained networks using built-in tools;
• Adapt existing networks to your imaging data;
• Quickly build new solutions to your own image analysis problems.

Orbit Image Analysis is a free open source software with the focus to quantify big images like whole slide scans.

It can connect to image servers, e.g. Omero.
Analysis can be done on your local computer or via scaleout functionality in a distrubuted computing environment like a Spark cluster.

Sophisticated image analysis algorithms incl. tissue quantification using machine learning, object segmentation and classification are build in. In addition a versatile API allows you to enhance Orbit and to run your own scripts.

automated open-source image acquisition and on-the-fly analysis pipeline (initially developped for analysis of mitotic defects in fission yeast)

"An open source machine learning framework for everyone "

TensorFlow™ is an open source software library for high performance numerical computation. Its flexible architecture allows easy deployment of computation across a variety of platforms (CPUs, GPUs, TPUs), and from desktops to clusters of servers to mobile and edge devices. Originally developed by researchers and engineers from the Google Brain team within Google’s AI organization, it comes with strong support for machine learning and deep learning and the flexible numerical computation core is used across many other scientific domains.

Super-resolve anisotropic EM data along low-res axis with deep learning.

SuRVoS: Super-Region Volume Segmentation workbench

A volume is first partitioned into Super-Regions (superpixels or supervoxels) and then interactively segmented by the user providing training annotations. SuRVoS can then learn from and extend the annotations to the whole volume.

WND-CHARM is a multi-purpose image classifier that can be applied to a wide variety of image classification tasks without modifications or fine-tuning, and yet provides classification accuracy comparable to state-of-the-art task-specific image classifiers. WND-CHARM can extract up to ~3,000 generic image descriptors (features) including polynomial decompositions, high contrast features, pixel statistics, and textures. These features are derived from the raw image, transforms of the image, and compound transforms of the image (transforms of transforms). The features are filtered and weighted depending on their effectiveness in discriminating between a set of predefined image classes (the training set). These features are then used to classify test images based on their similarity to the training classes. This classifier was tested on a wide variety of imaging problems including biological and medical image classification using several imaging modalities, face recognition, and other pattern recognition tasks. WND-CHARM is an acronym that stands for "Weighted Neighbor Distance using Compound Hierarchy of Algorithms Representing Morphology."

"we propose a novel automatic 3D neuron reconstruction algorithm, named Rivulet, which is based on the multi-stencils fast-marching and iterative back-tracking. The proposed Rivulet algorithm is capable of tracing discontinuous areas without being interrupted by densely distributed noises." 

This plugin can be used with default parameters or with user-defined parameters.

Example image obtained from Rivulet Wiki website (https://github.com/RivuletStudio/Rivulet-Neuron-Tracing-Toolbox/wiki) 

Advanced Cell Classifier is a data analyzer program to evaluate cell-based high-content screens and tissue section images developed at the Biological Research Centre, Szeged and FIMM, Helsinki (formerly at ETH Zurich). The basic aim is to provide a very accurate analysis with minimal user interaction using advanced machine learning methods.

QuPath is open source software for Quantitative Pathology. QuPath has been developed as a research tool at Queen's University Belfast.

This is the "prediction step" of the Pyxit segmentation model builder. It is a learnable segmentation algorithm based on ground-truth images and segmentation mask. It learns a multiple output pixel classification algorithm. It downloads from Cytomine-Core annotation images+alphamasks from project(s), build a segmentation (pixel classifier) model which is saved locally. Typical application: tumor detection in tissues in histology slides. 

This is a learnable segmentation algorithm based on ground-truth images and segmentation mask. It learns a multiple output pixel classification algorithm. It downloads from Cytomine-Core annotation images+alphamasks from project(s), build a segmentation (pixel classifier) model which is saved locally. Typical application: tumor detection in tissues in histology slides. It is based on "Fast Multi-Class Image Annotation with Random Subwindows and Multiple Output Randomized Trees" http://orbi.ulg.ac.be/handle/2268/12205 and was used in "A hybrid human-computer approach for large-scale image-based measurements using web services and machine learning" http://orbi.ulg.ac.be/handle/2268/162084?locale=en

This module is for applying classification models on objects. It downloads from Cytomine-Core annotation images and coordinate of annotated objects from project(s) and build a annotation classification model which is saved locally. It downloads from Cytomine-Core annotations images from an image (e.g. detected by an object finder), apply a classification model (previously saved locally), and uploads to Cytomine-Core annotation terms (in a userjob layer).

This module is for learning classification models from ground-truth data (supervised learning). It downloads from Cytomine-Core annotation images and coordinate of annotated objects from project(s) and build a annotation classification model which is saved locally.  

It is used by Cytomine DataMining applications: classification_validation, classification_model_builder, classification_prediction, segmentation_model_builder and segmentation_prediction. But it can be run without Cytomine on local data (using dir_ls and dir_ts arguments).

SLDC is an open-source Python workflow. SLDC stands for Segment Locate Dispatch Classify. This framework aims at facilitating the development of algorithms for detecting objects in multi-gigapixel images. Particularly, it provides algorithm developers with a structure to define problem-dependent components of their processing workflow (i.e. segmentation and classification) in a concise way. Every other concern such as parallelization and large image handling are encapsulated by the framework. It also features a powerful and customizable logging system and some components to apply several workflows one after another on a same image. SLDC can work on local images or interact with Cytomine

Example image:

Toy image data

The workflow consists of firstly identifying spot (which can be also gravity center of cells identified by another method), and then secondly compute trajectories by linking these spots by global optimisation with a cost function. This method is part of the methods evaluated in Chanouard et al (2014) as "method 9" and is described in detail in its supplementary PDF (page 65).

Dependencies

Following plugins are required.

1. JAR to be placed under IJ plugin directory
2. A pdf file with instructions and output description is also available in the zip .
3. MTrackJ : Used for visualization of tracks. Preinstalled in Fiji.
4. Imagescience.jar: This library is used by MTrackJ. Use update site to install this plugin.
5. jama.jar. Preinstalled in Fiji.

##Advantages:

• support blinking (with a parameters allowing it or not)
• fast,
• can be used in batch, some analysis results provided.
• No dynamic model.
• The tracking part is not dependent of ImageJ.

Pitfalls:

• does not support division
• the optimization algorithm used is a simulated annealing, so results can be slightly different between two runs.
• No Dynamic model (so less good results but can be used for a first study of the kind of movements)

##The sample data

The parameters used for this example data Beads, were

1. detection: 150
2. the max distance in pixels: 20
3. max allowed disappearance in frame: 1

An example macro introduced in the documentation page of the ImageJ plugin Trainable Weka Segmentation (in Fiji, it's bundled). A segmentation protocol based on machine learning. Full macro is available in the "Download" Link. 

This plugin can be trained to learn from the user input and perform later the same task in unknown (test) data. Weka: it makes use of all the powerful tools and classifiers from the latest version of Weka. Segmentation: it provides a labeled result based on the training of a chosen classifier. Trainable Weka Segmentation Complete macro example is at the end of the page.

CellDetector can detect cells (or other objects) in microscopy images such as histopathology, fluorescence, phase contrast, bright field, etc. It uses a machine learning-based method where a cell model is learned from simple dot annotations on a few images for training and predict on test sets. The installation requires some efforts but the instruction is well explained. Training parameters should be tuned for different datasets, but the default settings could be a good starting point.

This workflow is used to track multiple (appear/disappear, dividing and merging) objects in presumably big 2D+t or 3D+t datasets. It is best suitable for roundish objects or spots. Tracking is done through segmentation, which can be obtained from ilastik pixel classification, or imported from other tools. Users should provide a few object level labels, and the software predicts results on the rest of the image or new images with similar image characteristics. As a result, all objects get assigned random IDs at the first frame of the image sequence and all descendants in the same track (also children objects such as daughter cells) inherit this ID.

This macro batch processes all the 2D images (tif and jpg files) located in a user defined folder by calling Fiji Weka trainable segmentation to classify each pixel, and reports the areas of each class in a human readable results table. The classifier to be applied to each image should be previously trained on a representative image by an expert and exported to file (Save classifier) into the image folder to be processed.

ilastik is a simple, user-friendly tool for interactive image classification, segmentation and analysis. It is built as a modular software framework, which currently has workflows for automated (supervised) pixel- and object-level classification, automated and semi-automated object tracking, semi-automated segmentation and object counting without detection. Most analysis operations are performed lazily, which enables targeted interactive processing of data subvolumes, followed by complete volume analysis in offline batch mode. Using it requires no experience in image processing.

ilastik (the image learning, analysis, and segmentation toolkit) provides non-experts with a menu of pre-built image analysis workflows. ilastik handles data of up to five dimensions (time, 3D space, and spectral dimension). Its workflows provide an interactive experience to give the user immediate feedback on the quality of the results yielded by her chosen parameters and/or labelings.

The most commonly used workflow is pixel classification, which requires very little parameter tuning and instead offers a machine learning technique for segmenting an image based on local image features computed for each pixel.

Other workflows include:

Object classification: Similar to pixel classification, but classifies previously segmented objects by object characteristics in a subsequent step

Autocontext: This workflow improves the pixel classification workflow by running it in multiple stages and showing each pixel the results of the previous stage.

Carving: Semi-automated segmentation of 3D objects (e.g. neurons) based on user-provided seeds

Manual Tracking: Semi-automated cell tracking of 2D+time or 3D+time images based on manual annotations

Automated tracking: Fully-automated cell tracking of 2D+time or 3D+time images with some parameter tuning

Density Counting: Learned cell population counting based on interactively provided user annotation

Strengths: interactive, simple interface (for non-experts), few parameters, larger-than-RAM data, multi-dimensional data (time, 3D space, channel), headless operation, batch mode, parallelized computation, open source

Weaknesses: Pre-built workflows (not reconfigurable), no plugin system, visualization sometimes buggy, must import 3D data to HDF5, tracking requires an external CPLEX installation

Supported Formats: hdf5, tiff, jpeg, png, bmp, pnm, gif, hdr, exr, sif

This workflow estimates (densely distributed) object counts by the density of objects in the image without performing segmentation or object detection. Current version only works for 2D images of roundish objects with similar sizes on relatively homogeneous background. Users should provide a few labels of background and objects (especially on clustered objects), and the tool predicts the density of objects on the entire image. Counting is then estimated by integrating the density values on the whole image or specified rectangular regions of interests.

Semi-automated segmentation of 3D objects (e.g. neurons) based on user-provided seeds.

VIGRA is a free C++ and Python library that provides fundamental image processing and analysis algorithms. Its generic architecture allows it to be used in many different application contexts and ecosystems. It is designed as an intelligent library (using the C++ template mechanism) which allows users to write code at a fairly high level of abstraction and optimizes away the abstraction overhead upon compilation. It can therefore work efficiently on very large data and forms the basis of ilastik and CellCognition.

Strengths: open source, high quality algorithms, unlimited array dimension, arbitrary pixel types and number of channels, high speed, well tested, very flexible, easy-to-use Python bindings, support for many common file formats (including HDF5)

Limitations: no GUI, C++ not suitable for everyone, BioFormats not supported, parallelization requires external control

Images and Multi-dimensional Arrays: templated image data structures for arbitrary pixel types, fixed-size vectors multi-dimensional arrays for arbitrary high dimensions pre-instantiated images with many different scalar and vector valued pixel types (byte, short, int, float, double, complex, RGB, RGBA etc.) 2-dimensional image iterators, multi-dimensional iterators for arbitrary high dimensions, adapters for various image and array subsets

input/output of many image file formats: Windows BMP, GIF, JPEG, PNG, PNM, Sun Raster, TIFF (including 32bit integer, float, and double pixel types and multi-page TIFF), Khoros VIFF, HDR (high dynamic range), Andor SIF, OpenEXR input/output of images with transparency (alpha channel) into suitable file formats. comprehensive support for HDF5 (input/output of arrays in arbitrary dimensions)

continuous reconstruction of discrete images using splines: Just create a SplineImageView of the desired order and access interpolated values and derivative at any real-valued coordinate.

Image Processing: STL-style image processing algorithms with functors (e.g. arithmetic and algebraic operations, gamma correction, contrast adaptation, thresholding), arbitrary regions of interest using mask images image resizing using resampling, linear interpolation, spline interpolation etc.

geometric transformations: rotation, mirroring, arbitrary affine transformations automated functor creation using expression templates

color space conversions: RGB, sRGB, R'G'B', XYZ, Lab*, Luv*, Y'PbPr, Y'CbCr, Y'IQ, and Y'UV real and complex Fourier transforms in arbitrary dimensions, cosine and sine transform (via fftw) noise normalization according to Förstner computation of the camera magnitude transfer function (MTF) via the slanted edge technique (ISO standard 12233)

Filters: 2-dimensional and separable convolution, Gaussian filters and their derivatives, Laplacian of Gaussian, sharpening etc. separable convolution and FFT-based convolution for arbitrary dimensional data resampling convolution (input and output image have different size) recursive filters (1st and 2nd order), exponential filters non-linear diffusion (adaptive filters), hourglass filter total-variation filtering and denoising (standard, higer-order, and adaptive methods)

tensor image processing: structure tensor, boundary tensor, gradient energy tensor, linear and non-linear tensor smoothing, eigenvalue calculation etc. (2D and 3D) distance transform (Manhattan, Euclidean, Checker Board norms, 2D and 3D) morphological filters and median (2D and 3D) Loy/Zelinsky symmetry transform Gabor filters

Segmentation: edge detectors: Canny, zero crossings, Shen-Castan, boundary tensor corner detectors: corner response function, Beaudet, Rohr and Förstner corner detectors tensor based corner and junction operators

region growing: seeded region growing, watershed algorithm

Image Analysis: connected components labeling (2D and 3D) detection of local minima/maxima (including plateaus, 2D and 3D) tensor-basesd image analysis (2D and 3D) powerful incremental computation of region and object statistics

3-dimensional Image Processing and Analysis: point-wise transformations, projections and expansions in arbitrary high dimensions all functors (e.g. regions statistics) readily apply to higher dimensional data as well separable convolution and FFT-based convolution filters, resizing, morphology, and Euclidean distance transform for arbitrary dimensional arrays (not just 3D) connected components labeling, seeded region growing, watershed algorithm for volume data

Machine Learning: random forest classifier with various tree building strategies variable importance, feature selection (based on random forest) unsupervised decomposition: PCA (principle component analysis) and pLSA (probabilistic latent semantic analysis)

Mathematical Tools: special functions (error function, splines of arbitrary order, integer square root, chi square distribution, elliptic integrals) random number generation rational and fixed point numbers quaternions polynomials and polynomial root finding matrix classes, linear algebra, solution of linear systems, eigen system computation, singular value decomposition

optimization: linear least squares, ridge regression, L1-constrained least squares (LASSO, non-negative LASSO, least angle regression), quadratic programming

Inter-language support: Python bindings in both directions (use Python arrays in C++, call VIGRA functions from Python) Matlab bindings of some functions

Image segmentation by representative colors selection. Two versions are available :

• thresholding
• positive and negative colors selection and SVM learning