Machine learning

BIIGLE is a web-based software for image and video annotation that enables collaborative research on large datasets. It offers tools for manual and computer-assisted annotation, quality control and the collaboration on custom taxonomies to describe objects. BIIGLE is freely available and can be installed in cloud environments, a local network or on mobile platforms during research expeditions. The public instance on biigle.de is free for non-commercial use.

MATLAB app to characterize nanoparticles imaged with super-resolution microscopy. nanoFeatures will read text and csv files from the NIKON and ONI microscopes and from the ThunderSTORM Fiji plugin, then cluster the localizations and filter by size and sphericity and finally output nanoparticle features like size, aspect ratio, and number of localizations per cluster (total and for each channel).

ELEPHANT is a platform for 3D cell tracking, based on incremental and interactive deep learning.
It implements a client-server architecture. The server is built as a web application that serves deep learning-based algorithms. The client application is implemented by extending Mastodon, providing a user interface for annotation, proofreading and visualization.

ZeroCostDL4Mic: exploiting Google Colab to develop a free and open-source toolbox for Deep-Learning in microscopy

ZeroCostDL4Mic is a collection of self-explanatory Jupyter Notebooks for Google Colab that features an easy-to-use graphical user interface. They are meant to quickly get you started on learning to use deep-learning for microscopy. 

BiaPy is an open source Python library for building bioimage analysis pipelines. 

BiaPy contains ready-to-use solutions for the tasks of semantic segmentation, instance segmentation, object detection, image denoising, single image super-resolution, self-supervised learning and image classification. 

btrack is a Python library for multi object tracking, used to reconstruct trajectories in crowded fields. btrack implemented a residual U-Net model coupledd with a classification CNN to allow accurate instance segmentation of the cell nuclei. To track the cells over time and through cell divisions, btrack developed a Bayesian cell tracking methodology that uses input features from the images to enable the retrieval of multi-generational lineage information from a corpus of thousands of hours of live-cell imaging data.

Open source deep learning based framework for multi-animal pose tracking. It can track animal and any number of animals and has a labeling/training GUI for learning and proofreading.

Algorithm and software created to extract animal trajectories from videos of a collection of animals up to 100 individuals. Idtrackerai uses two convolutional networks: one for animal identification and another to detect when animals touch or cross each other.

The method proposed in this paper is a robust combination of multi-task learning and unsupervised domain adaptation for segmenting amoeboid cells in microscopy. This end-to-end framework provides a consolidated mechanism to harness the potential of multi-task learning to isolate and segment clustered cells from low contrast brightfield images, and it simultaneously leverages deep domain adaptation to segment fluorescent cells without explicit pixel-level re- annotation of the data.

The entry-point to the codebase is the main.py file. The user has the option to

• Train the network on their own dataset
• Load a pre-trained model and use that for inference on their own data
• Note: The provided pretrained model was trained on 256x256 images. Results on different resolutions could require fine-tuning This model is trained (supervised) on brightfield, and domain adapted to fluorescence data. The results are saved as 'inference.png'

This workflow describes a deep-learning based pipeline for reliable single-organoid segmentation and tracking in 2D+t high-resolution brightfield microscopy of mouse mammary epithelial organoids. The pipeline involves a four-layer U-Net to infer semantic segmentation predictions, adaptive morphological filtering to establish candidate organoid instances, and a shape-similarity-constrained, instance-segmentation-correcting tracking step to associate the corresponding organoid instances in time.

It is particularly focused on automatically detecting an organoid located approximately in the center of the first frame and track all its subsequent instances in the remaining frames, emphasizing on accurate organoid boundary delineation. Furthermore, segmentation network was trained using plausible pix2pixHD-generated bioimage data. Syntheric image simulator code and data are also available here.

OrganoID is an image analysis platform that automatically recognizes, labels, and tracks single organoids, pixel-by-pixel, in brightfield and phase-contrast microscopy experiments. The platform was trained on images of pancreatic cancer organoids and validated on separate images of pancreatic, lung, colon, and adenoid cystic carcinoma organoids.

Fractal is a framework to process high-content imaging data at scale and prepare it for interactive visualization. Fractal provides distributed workflows that convert TBs of image data into OME-Zarr files. The platform then processes the 3D image data by applying tasks like illumination correction, maximum intensity projection, 3D segmentation using cellpose and measurements using napari workflows. The pyramidal OME-Zarr files enable interactive visualization in the napari viewer.

This workflow applies a Stardist pre-trained model (versatile_fluo or versatile_HE) depending on the input images ie. uses both models for a dataset including both fluorescence (grayscale or RGB where all channels are equal) and H&E stained (RGB where channels are not equal) images.

This version uses tensorflow CPU version (See Dockerfile) to ensure compatibility with a larger number of computers. A GPU version should be possible by adapting the Dockerfile with tensorflow-gpu and/or nvidia-docker images.

This workflow processes a group of images containing cells with discernible nuclei and segments the nuclei and outputs a binary mask that show where nuclei were detected. It performs 2D nuclei segmentation using pre-trained nuclei segmentation models of Cellpose. And it was developed as a test workflow for Neubias BIAFLOWS Benchmarking tool.

The Incucyte® Base Analysis Software provides a guided interface and purpose-built tools, which include the process of acquiring, viewing, analyzing and sharing images of living cells.

The empanada-napari plugin is built to democratize deep learning image segmentation for researchers in electron microscopy (EM). It ships with MitoNet, a generalist model for the instance segmentation of mitochondria. There are also tools to quickly build and annotate training datasets, train generic panoptic segmentation models, finetune existing models, and scalably run inference on 2D or 3D data. To make segmentation model training faster and more robust, CEM pre-trained weights are used by default. These weights were trained using an unsupervised learning algorithm on over 1.5 million EM images from hundreds of unique EM datasets making them remarkably general.

AnnotatorJ is a Fiji Plugin to ease annotation of images, particulrly useful for Deep Learning or to validate an alogorithm. Interestingly, it allows annotation for instance segmentation, semantic segmentation, or bounding box annotations. It includes toolssuch as active contours to ease these annotations.

Machine Learning made easy

APEER ML provides an easy way to train your own machine learning
models and segment your microscopy images. No expertise or coding required.

Phindr3D is a comprehensive shallow-learning framework for automated quantitative phenotyping of three-dimensional (3D) high content screening image data using unsupervised data-driven voxel-based feature learning, which enables computationally facile classification, clustering and data visualization.

Please see our GitHub page and the original publication for details.

Deep learning based image restoration methods have recently been made available to restore images from under-exposed imaging conditions, increase spatio-temporal resolution (CARE) or self-supervised image denoising (Noise2Void). These powerful methods outperform conventional state-of-the-art methods and leverage down-stream analyses significantly such as segmentation and quantification.

To bring these new tools to a broader platform in the image analysis community, we developed a simple Jupyter based graphical user interface for CARE and Noise2Void, which lowers the burden for non-programmers and biologists to access these powerful methods in their daily routine.  CARE-less supports temporal, multi-channel image and volumetric data and many file formats by using the bioformats library. The user is guided through the different computation steps via inline documentation. For standard use cases, the graphical user interface exposes the most relevant parameters such as patch size and number of training iterations, while expert users still have access to advanced parameters such as U-net depth and kernel sizes. In addition, CARE-less provides visual outputs for training convergence and restoration quality. Any project settings can be stored and reused from command line for processing on compute clusters. The generated output files preserve important meta-data such as pixel sizes, axial spacing and time intervals.

This tool allows the user to define structures of interest by interactively marking a subset of pixels. Thanks to the real-time feedback, the user can place new markings strategically, depending on the current outcome.

BioImage.IO -- a collaborative effort to bring AI models to the bioimaging community. 

• Integrated with Fiji, ilastik, ImJoy and more
• Try model instantly with BioEngine
• Contribute your models via Github

This is a database of pretrained deep Learning models. 

Yet another pixel classifier Yapic is a deep learning tool to :

train your own filter to enhance the structure of your choice 

train multiple filter at once 

it is based on the u-net convolutional filter . 

To train it : annotation can come from example from Ilastik software , tif labelled files can be transferred to yapic. 

Training takes about hours to days , prediction takes seconds once trained .

It can be ran from command line .

note that only 10 to 20 images with sparse labeling are required for efficient training 

LOBSTER (Little Objects Segmentation and Tracking Environment), an environment designed to help scientists design and customize image analysis workflows to accurately characterize biological objects from a broad range of fluorescence microscopy images, including large images, i.e. terabytes of data, exceeding workstation main memory.

• 75 workflows available 
• no programming, with GUI
• matlab based 

Web based application for deep learning cell classifier . Aim to replace cellprofiler analyst and advanced cell classifier.

This is the ImageJ/Fiji plugin for StarDist, a cell/nuclei detection method for microscopy images with star-convex shape priors ( typically for Dapi like staining of nuclei). The plugin can be used to apply already trained models to new images.

Summary

Deep learning-based segmentation of cells, both fluorescence, and bright-field images ("a generalist algorithm for cellular segmentation"). The tool can be used either online or local or via notebooks (e.g. ZeroCostDL4Mic).

How to use it

cellpose can be used online via ready-to-use Jyupyter notebooks with very good documentation. These notebooks are listed here.

Local Installation

The general local installation procedure can be found here.

... Installing to Silicon Mac (M1 processor) needs some tricks, and as of October 2021, the following sequence of commands works. numba should be conda-installed before pip-installing the cellpose.

conda create --name cellpose python=3.8
conda activate cellpose
conda install numba
git clone https://github.com/MouseLand/cellpose.git
cd cellpose
pip install -e .

DeepImageJ is a user-friendly plugin that enables the use of a variety of pre-trained deep learning models in ImageJ and Fiji. The plugin bridges the gap between deep learning and standard life-science applications. DeepImageJ runs image-to-image operations on a standard CPU-based computer and does not require any deep learning expertise.

Training developper constructs and upload trained model, and made them available to users.

Models are available in a repository here.

It is macro recordable. It is advised to use DeepImageJ on a computer with GPU (CPU will likely be 20x slower)

Rivuletpy dockerised workflow for BIAFLOWS.

Cell tracking using MU-Lux-CZ algorithm. Dockerized Workflow for BIAFLOWS implemented by Martin Maska (Masaryk University).

Track non-dividing particles in 2D time-lapse image.

Execute Nuclei Segmentation in 3D images using pixel classification with ilastik.

U-Net segmentation as presented in Reference Publication. The model predicts three classes: background, edge and foreground. The model was trained with Kaggle Data Science Bowl (DSB) 2018 training set.

Nuclei Segmentation using Deep Learning for individual cell analysis (DeepCell).

Wavelet Toolbox™ provides apps and functions for the time-frequency analysis of signals and multiscale analysis of images.

ASAP allows to automatically detect, classify and quantify structures acquired by super resolution microscopy. 

This plugin ships automated methods for extracting trajectories of multiples objects in a sequence of 2D or 3D images. Up to version 2 it was known as the ‘Probabilistic particle tracker’ plugin.

Wolfram Mathematica (usually termed Mathematica) is a modern technical computing system spanning most areas of technical computing — including neural networks, machine learning, image processing, geometry, data science, visualizations, and others. The system is used in many technical, scientific, engineering, mathematical, and computing fields.

Dragonfly is a software platform for the intuitive inspection of multi-scale multi-modality image data. Its user-friendly experience translates into powerful quantitative findings with high-impact visuals, driven by nuanced easy-to-learn controls.

For segmentation: It provides an engine fior machine Learning, Watershed and superpixel methods, support histological data .

It offers a 3D viewer, and python scripting capacities .

It is free for reserach use, but not for commercial usage.

online image data management system which supports authenticated image upload, cloud-based storage, project-based management and viewing of standard and whole slide images. One can use different annotation tools to highlight important objects or areas within images. It is the first basic version and new features such as sharing for easy collaboration with your colleagues or first automated analysis applications based on artificial intelligence will be added soon.

Ikosa Portal: multi user image data management

Ikosa Prisma: Automated Image Analysis based on deep learning (available in summer 2019)

Free if limited to 2 users and 1 gigabyte, otherwise montly fees.

NEUBIAS-WG5 workflow for nuclei segmentation using ilastik v1.3.2 and Python post-processing.

NEUBIAS-WG5 workflow for nuclei segmentation using Mask-RCNN. The workflow uses Matterport Mask-RCNN. Keras implementation. The model was trained with Kaggle 2018 Data Science Bowl images.

This workflow predict landmark positions on images by using DMBL landmark detection models.

An implementation of Belief Propagation for factor graphs, also known as the sum-product algorithm

This workflow predict landmark positions on images by using LC landmark detection models.

This workflow trains LC landmark detection models from a dataset of annotated images.

This workflow predict landmark positions on images by using MSET landmark detection models.

This workflow trains MSET landmark detection models from a dataset of annotated images.

PyTorch is an open-source machine learning library for Python, based on Torch, used for applications such as natural language processing.

Collection of several basic standard image segmentation methods focusing on medical imaging. In particular, the key block/applications are (un)supervised image segmentation using superpixels, object centre detection and region growing with a shape prior. Besides the open-source code, there is also a few sample images.

ImJoy is a plugin powered hybrid computing platform for deploying deep learning applications such as advanced image analysis tools.

ImJoy runs on mobile and desktop environment cross different operating systems, plugins can run in the browser, localhost, remote and cloud servers.

With ImJoy, delivering Deep Learning tools to the end users is simple and easy thanks to its flexible plugin system and sharable plugin URL. Developer can easily add rich and interactive web interfaces to existing Python code.

This workflow describes a semi-automatic image segmentation procedure for 3D reconstructions of the coronary arterial tree, after which how different morphometric features are automatically extracted, including vessel lumen diameter of the three main coronaries.

Labkit is an open-source tool to segment truly large image data using sparse training data. It has an intuitive and responsive user interface based on Big Data Viewer, allowing users to conveniently browse and annotate even terabyte sized image volumes.

Update site: Labkit

Scikit-learn (sklearn) is a python library used for machine learning. sklearn contains simple and efficient tools for data mining and data analysis. Modules and functions include those for classification, regression, clustering, dimensionality reduction, model selection and data preprocessing. Many people have contributed to sklearn (list of authors)

3-D density kernel estimation (DKE-3-D) method, utilises an ensemble of random decision trees for counting objects in 3D images. DKE-3-D avoids the problem of discrete object identification and segmentation, common to many existing 3-D counting techniques, and outperforms other methods when quantification of densely packed and heterogeneous objects is desired. 

Please also check out ImageJ and Fiji.

Biocat is a java based software that allows to perform image classification or segmentation using machine learning. Several algorithm for the classification are available.

ZEN and APEER – Open Ecosystem for integrated Machine-Learning Workflows

Open ecosystem for integrated machine-learning workflows to train and use machine-learning models for image processing and image analysis inside the ZEN software or on the APEER cloud-based platform

Highlights ZEN

• Simple User Interface for Labeling and Training
• Engineered Features Sets and Deep Feature Extraction + Random Forrest for Semantic Segmentation
• Object Classification workflows
• Probability Thresholds and Conditional Random Fields
• Import your own trained models as *.czann files (see: czmodel · PyPI)
• Import "AIModel Containes" from arivis AI for advanced Instance Segmentation
• Integration into ZEN Measurement Framework
• Support for Multi-dimensional Datasets and Tile Images
• open and standardized format to store trained models

ZEN Intellesis Segmentation

ZEN Intellesis - Pretrained Networks

Intellesis Object Classification

Highlights Aarivis AI

• Web-based tool to label datasets to train Deep Neural Networks
• Fully automated hyper-parameter tuning
• Export of trained models for semantic segmentation and AIModelContainer for Instance Segmentation

Annotation Tool

Chainer is a Python-based deep learning framework aiming at flexibility. It provides automatic differentiation APIs based on the define-by-run approach (a.k.a. dynamic computational graphs) as well as object-oriented high-level APIs to build and train neural networks. It also supports CUDA/cuDNN using CuPy for high performance training and inference. For more details of Chainer, see the documents and resources listed above and join the community in Forum, Slack, and Twitter.

Python is a programming language.

Python 2.7.0 was released on July 3rd, 2010.

Python 2.7 is scheduled to be the last major version in the 2.x series before it moves into an extended maintenance period. This release contains many of the features that were first released in Python 3.1.

 A bugfix release, 2.7.16, is currently available. Its use is recommended.

Quantitative Criterion Acquisition Network (QCA Net) performs instance segmentation of 3D fluorescence microscopic images. QCA Net consists of Nuclear Segmentation Network (NSN) that learned nuclear segmentation task and Nuclear Detection Network (NDN) that learned nuclear identification task. QCA Net performs instance segmentation of the time-series 3D fluorescence microscopic images at each time point, and the quantitative criteria for mouse development are extracted from the acquired time-series segmentation image. The detailed information on this program is described in our manuscript posted on bioRxiv.

Python is a programming language.

An ImageJ plugin for DEFCoN, the fluorescence spot counter based on fully convolutional neural networks

NiftyNet is a TensorFlow-based open-source convolutional neural networks (CNNs) platform for research in medical image analysis and image-guided therapy. NiftyNet’s modular structure is designed for sharing networks and pre-trained models. Using this modular structure you can:

• Get started with established pre-trained networks using built-in tools;
• Adapt existing networks to your imaging data;
• Quickly build new solutions to your own image analysis problems.

Orbit Image Analysis is a free open source software with the focus to quantify big images like whole slide scans.

It can connect to image servers, e.g. Omero.
Analysis can be done on your local computer or via scaleout functionality in a distrubuted computing environment like a Spark cluster.

Sophisticated image analysis algorithms incl. tissue quantification using machine learning, object segmentation and classification are build in. In addition a versatile API allows you to enhance Orbit and to run your own scripts.

automated open-source image acquisition and on-the-fly analysis pipeline (initially developped for analysis of mitotic defects in fission yeast)

"An open source machine learning framework for everyone "

TensorFlow™ is an open source software library for high performance numerical computation. Its flexible architecture allows easy deployment of computation across a variety of platforms (CPUs, GPUs, TPUs), and from desktops to clusters of servers to mobile and edge devices. Originally developed by researchers and engineers from the Google Brain team within Google’s AI organization, it comes with strong support for machine learning and deep learning and the flexible numerical computation core is used across many other scientific domains.

Super-resolve anisotropic EM data along low-res axis with deep learning.

SuRVoS: Super-Region Volume Segmentation workbench

A volume is first partitioned into Super-Regions (superpixels or supervoxels) and then interactively segmented by the user providing training annotations. SuRVoS can then learn from and extend the annotations to the whole volume.

WND-CHARM is a multi-purpose image classifier that can be applied to a wide variety of image classification tasks without modifications or fine-tuning, and yet provides classification accuracy comparable to state-of-the-art task-specific image classifiers. WND-CHARM can extract up to ~3,000 generic image descriptors (features) including polynomial decompositions, high contrast features, pixel statistics, and textures. These features are derived from the raw image, transforms of the image, and compound transforms of the image (transforms of transforms). The features are filtered and weighted depending on their effectiveness in discriminating between a set of predefined image classes (the training set). These features are then used to classify test images based on their similarity to the training classes. This classifier was tested on a wide variety of imaging problems including biological and medical image classification using several imaging modalities, face recognition, and other pattern recognition tasks. WND-CHARM is an acronym that stands for "Weighted Neighbor Distance using Compound Hierarchy of Algorithms Representing Morphology."

"we propose a novel automatic 3D neuron reconstruction algorithm, named Rivulet, which is based on the multi-stencils fast-marching and iterative back-tracking. The proposed Rivulet algorithm is capable of tracing discontinuous areas without being interrupted by densely distributed noises." 

This plugin can be used with default parameters or with user-defined parameters.

Example image obtained from Rivulet Wiki website (https://github.com/RivuletStudio/Rivulet-Neuron-Tracing-Toolbox/wiki) 

Advanced Cell Classifier is a data analyzer program to evaluate cell-based high-content screens and tissue section images developed at the Biological Research Centre, Szeged and FIMM, Helsinki (formerly at ETH Zurich). The basic aim is to provide a very accurate analysis with minimal user interaction using advanced machine learning methods.

QuPath is open source software for Quantitative Pathology. QuPath has been developed as a research tool at Queen's University Belfast.

This is the "prediction step" of the Pyxit segmentation model builder. It is a learnable segmentation algorithm based on ground-truth images and segmentation mask. It learns a multiple output pixel classification algorithm. It downloads from Cytomine-Core annotation images+alphamasks from project(s), build a segmentation (pixel classifier) model which is saved locally. Typical application: tumor detection in tissues in histology slides. 

This is a learnable segmentation algorithm based on ground-truth images and segmentation mask. It learns a multiple output pixel classification algorithm. It downloads from Cytomine-Core annotation images+alphamasks from project(s), build a segmentation (pixel classifier) model which is saved locally. Typical application: tumor detection in tissues in histology slides. It is based on "Fast Multi-Class Image Annotation with Random Subwindows and Multiple Output Randomized Trees" http://orbi.ulg.ac.be/handle/2268/12205 and was used in "A hybrid human-computer approach for large-scale image-based measurements using web services and machine learning" http://orbi.ulg.ac.be/handle/2268/162084?locale=en

This module is for applying classification models on objects. It downloads from Cytomine-Core annotation images and coordinate of annotated objects from project(s) and build a annotation classification model which is saved locally. It downloads from Cytomine-Core annotations images from an image (e.g. detected by an object finder), apply a classification model (previously saved locally), and uploads to Cytomine-Core annotation terms (in a userjob layer).

This module is for learning classification models from ground-truth data (supervised learning). It downloads from Cytomine-Core annotation images and coordinate of annotated objects from project(s) and build a annotation classification model which is saved locally.  

It is used by Cytomine DataMining applications: classification_validation, classification_model_builder, classification_prediction, segmentation_model_builder and segmentation_prediction. But it can be run without Cytomine on local data (using dir_ls and dir_ts arguments).

SLDC is an open-source Python workflow. SLDC stands for Segment Locate Dispatch Classify. This framework aims at facilitating the development of algorithms for detecting objects in multi-gigapixel images. Particularly, it provides algorithm developers with a structure to define problem-dependent components of their processing workflow (i.e. segmentation and classification) in a concise way. Every other concern such as parallelization and large image handling are encapsulated by the framework. It also features a powerful and customizable logging system and some components to apply several workflows one after another on a same image. SLDC can work on local images or interact with Cytomine

Example image:

Toy image data

The workflow consists of firstly identifying spot (which can be also gravity center of cells identified by another method), and then secondly compute trajectories by linking these spots by global optimisation with a cost function. This method is part of the methods evaluated in Chanouard et al (2014) as "method 9" and is described in detail in its supplementary PDF (page 65).

Dependencies

Following plugins are required.

1. JAR to be placed under IJ plugin directory
2. A pdf file with instructions and output description is also available in the zip .
3. MTrackJ : Used for visualization of tracks. Preinstalled in Fiji.
4. Imagescience.jar: This library is used by MTrackJ. Use update site to install this plugin.
5. jama.jar. Preinstalled in Fiji.

##Advantages:

• support blinking (with a parameters allowing it or not)
• fast,
• can be used in batch, some analysis results provided.
• No dynamic model.
• The tracking part is not dependent of ImageJ.

Pitfalls:

• does not support division
• the optimization algorithm used is a simulated annealing, so results can be slightly different between two runs.
• No Dynamic model (so less good results but can be used for a first study of the kind of movements)

##The sample data

The parameters used for this example data Beads, were

1. detection: 150
2. the max distance in pixels: 20
3. max allowed disappearance in frame: 1

An example macro introduced in the documentation page of the ImageJ plugin Trainable Weka Segmentation (in Fiji, it's bundled). A segmentation protocol based on machine learning. Full macro is available in the "Download" Link. 

This plugin can be trained to learn from the user input and perform later the same task in unknown (test) data. Weka: it makes use of all the powerful tools and classifiers from the latest version of Weka. Segmentation: it provides a labeled result based on the training of a chosen classifier. Trainable Weka Segmentation Complete macro example is at the end of the page.

CellDetector can detect cells (or other objects) in microscopy images such as histopathology, fluorescence, phase contrast, bright field, etc. It uses a machine learning-based method where a cell model is learned from simple dot annotations on a few images for training and predict on test sets. The installation requires some efforts but the instruction is well explained. Training parameters should be tuned for different datasets, but the default settings could be a good starting point.

This workflow is used to track multiple (appear/disappear, dividing and merging) objects in presumably big 2D+t or 3D+t datasets. It is best suitable for roundish objects or spots. Tracking is done through segmentation, which can be obtained from ilastik pixel classification, or imported from other tools. Users should provide a few object level labels, and the software predicts results on the rest of the image or new images with similar image characteristics. As a result, all objects get assigned random IDs at the first frame of the image sequence and all descendants in the same track (also children objects such as daughter cells) inherit this ID.

This macro batch processes all the 2D images (tif and jpg files) located in a user defined folder by calling Fiji Weka trainable segmentation to classify each pixel, and reports the areas of each class in a human readable results table. The classifier to be applied to each image should be previously trained on a representative image by an expert and exported to file (Save classifier) into the image folder to be processed.

ilastik is a simple, user-friendly tool for interactive image classification, segmentation and analysis. It is built as a modular software framework, which currently has workflows for automated (supervised) pixel- and object-level classification, automated and semi-automated object tracking, semi-automated segmentation and object counting without detection. Most analysis operations are performed lazily, which enables targeted interactive processing of data subvolumes, followed by complete volume analysis in offline batch mode. Using it requires no experience in image processing.

ilastik (the image learning, analysis, and segmentation toolkit) provides non-experts with a menu of pre-built image analysis workflows. ilastik handles data of up to five dimensions (time, 3D space, and spectral dimension). Its workflows provide an interactive experience to give the user immediate feedback on the quality of the results yielded by her chosen parameters and/or labelings.

The most commonly used workflow is pixel classification, which requires very little parameter tuning and instead offers a machine learning technique for segmenting an image based on local image features computed for each pixel.

Other workflows include:

Object classification: Similar to pixel classification, but classifies previously segmented objects by object characteristics in a subsequent step

Autocontext: This workflow improves the pixel classification workflow by running it in multiple stages and showing each pixel the results of the previous stage.

Carving: Semi-automated segmentation of 3D objects (e.g. neurons) based on user-provided seeds

Manual Tracking: Semi-automated cell tracking of 2D+time or 3D+time images based on manual annotations

Automated tracking: Fully-automated cell tracking of 2D+time or 3D+time images with some parameter tuning

Density Counting: Learned cell population counting based on interactively provided user annotation

Strengths: interactive, simple interface (for non-experts), few parameters, larger-than-RAM data, multi-dimensional data (time, 3D space, channel), headless operation, batch mode, parallelized computation, open source

Weaknesses: Pre-built workflows (not reconfigurable), no plugin system, visualization sometimes buggy, must import 3D data to HDF5, tracking requires an external CPLEX installation

Supported Formats: hdf5, tiff, jpeg, png, bmp, pnm, gif, hdr, exr, sif

This workflow estimates (densely distributed) object counts by the density of objects in the image without performing segmentation or object detection. Current version only works for 2D images of roundish objects with similar sizes on relatively homogeneous background. Users should provide a few labels of background and objects (especially on clustered objects), and the tool predicts the density of objects on the entire image. Counting is then estimated by integrating the density values on the whole image or specified rectangular regions of interests.

Semi-automated segmentation of 3D objects (e.g. neurons) based on user-provided seeds.

VIGRA is a free C++ and Python library that provides fundamental image processing and analysis algorithms. Its generic architecture allows it to be used in many different application contexts and ecosystems. It is designed as an intelligent library (using the C++ template mechanism) which allows users to write code at a fairly high level of abstraction and optimizes away the abstraction overhead upon compilation. It can therefore work efficiently on very large data and forms the basis of ilastik and CellCognition.

Strengths: open source, high quality algorithms, unlimited array dimension, arbitrary pixel types and number of channels, high speed, well tested, very flexible, easy-to-use Python bindings, support for many common file formats (including HDF5)

Limitations: no GUI, C++ not suitable for everyone, BioFormats not supported, parallelization requires external control

Images and Multi-dimensional Arrays: templated image data structures for arbitrary pixel types, fixed-size vectors multi-dimensional arrays for arbitrary high dimensions pre-instantiated images with many different scalar and vector valued pixel types (byte, short, int, float, double, complex, RGB, RGBA etc.) 2-dimensional image iterators, multi-dimensional iterators for arbitrary high dimensions, adapters for various image and array subsets

input/output of many image file formats: Windows BMP, GIF, JPEG, PNG, PNM, Sun Raster, TIFF (including 32bit integer, float, and double pixel types and multi-page TIFF), Khoros VIFF, HDR (high dynamic range), Andor SIF, OpenEXR input/output of images with transparency (alpha channel) into suitable file formats. comprehensive support for HDF5 (input/output of arrays in arbitrary dimensions)

continuous reconstruction of discrete images using splines: Just create a SplineImageView of the desired order and access interpolated values and derivative at any real-valued coordinate.

Image Processing: STL-style image processing algorithms with functors (e.g. arithmetic and algebraic operations, gamma correction, contrast adaptation, thresholding), arbitrary regions of interest using mask images image resizing using resampling, linear interpolation, spline interpolation etc.

geometric transformations: rotation, mirroring, arbitrary affine transformations automated functor creation using expression templates

color space conversions: RGB, sRGB, R'G'B', XYZ, Lab*, Luv*, Y'PbPr, Y'CbCr, Y'IQ, and Y'UV real and complex Fourier transforms in arbitrary dimensions, cosine and sine transform (via fftw) noise normalization according to Förstner computation of the camera magnitude transfer function (MTF) via the slanted edge technique (ISO standard 12233)

Filters: 2-dimensional and separable convolution, Gaussian filters and their derivatives, Laplacian of Gaussian, sharpening etc. separable convolution and FFT-based convolution for arbitrary dimensional data resampling convolution (input and output image have different size) recursive filters (1st and 2nd order), exponential filters non-linear diffusion (adaptive filters), hourglass filter total-variation filtering and denoising (standard, higer-order, and adaptive methods)

tensor image processing: structure tensor, boundary tensor, gradient energy tensor, linear and non-linear tensor smoothing, eigenvalue calculation etc. (2D and 3D) distance transform (Manhattan, Euclidean, Checker Board norms, 2D and 3D) morphological filters and median (2D and 3D) Loy/Zelinsky symmetry transform Gabor filters

Segmentation: edge detectors: Canny, zero crossings, Shen-Castan, boundary tensor corner detectors: corner response function, Beaudet, Rohr and Förstner corner detectors tensor based corner and junction operators

region growing: seeded region growing, watershed algorithm

Image Analysis: connected components labeling (2D and 3D) detection of local minima/maxima (including plateaus, 2D and 3D) tensor-basesd image analysis (2D and 3D) powerful incremental computation of region and object statistics

3-dimensional Image Processing and Analysis: point-wise transformations, projections and expansions in arbitrary high dimensions all functors (e.g. regions statistics) readily apply to higher dimensional data as well separable convolution and FFT-based convolution filters, resizing, morphology, and Euclidean distance transform for arbitrary dimensional arrays (not just 3D) connected components labeling, seeded region growing, watershed algorithm for volume data

Machine Learning: random forest classifier with various tree building strategies variable importance, feature selection (based on random forest) unsupervised decomposition: PCA (principle component analysis) and pLSA (probabilistic latent semantic analysis)

Mathematical Tools: special functions (error function, splines of arbitrary order, integer square root, chi square distribution, elliptic integrals) random number generation rational and fixed point numbers quaternions polynomials and polynomial root finding matrix classes, linear algebra, solution of linear systems, eigen system computation, singular value decomposition

optimization: linear least squares, ridge regression, L1-constrained least squares (LASSO, non-negative LASSO, least angle regression), quadratic programming

Inter-language support: Python bindings in both directions (use Python arrays in C++, call VIGRA functions from Python) Matlab bindings of some functions

Image segmentation by representative colors selection. Two versions are available :

• thresholding
• positive and negative colors selection and SVM learning

The Trainable Weka Segmentation is a Fiji plugin that combines a collection of machine learning algorithms with a set of selected image features to produce pixel-based segmentations. Weka (Waikato Environment for Knowledge Analysis) can itself be called from the plugin. It contains a collection of visualization tools and algorithms for data analysis and predictive modeling, together with graphical user interfaces for easy access to this functionality. As described on their wikipedia site, the advantages of Weka include: - freely availability under the GNU General Public License - portability, since it is fully implemented in the Java programming language and thus runs on almost any modern computing platform - a comprehensive collection of data preprocessing and modeling techniques - ease of use due to its graphical user interfaces - Weka supports several standard data mining tasks, more specifically, data preprocessing, clustering, classification, regression, visualization, and feature selection.

The main goal of this plugin is to work as a bridge between the Machine Learning and the Image Processing fields. It provides the framework to use and, more important, compare any available classifier to perform image segmentation based on pixel classification.