This is a Fiji plugin based on the SIOX projectto segment color images. SIOX stands for Simple Interactive Object Extraction, a method for extracting foreground from still images with very little user interaction. SIOX is fast, noise robust and can therefore also be used for the segmentation of videos. It avoids many of the drawbacks of graph-based segmentation methods but performs about equally well on different benchmarks. SIOX is open and free (Apache License) and the authors have intentionally not patented any part of the technology. As a result, it has been integrated into several open-source image manipulation programs over the past years. SIOX is the underlying algorithm of the foreground extraction tool in the GNU Image Manipulation Program (GIMP) and is part of the tracer tool in Inkscape. SIOX originates from E-Chalk where an instructor standing in front of an electronic chalkboard is segmented. Variants of SIOX are being used for robotic vision and for improving 3D time-of-flight camera segmentation.

Quick Start

In order to call the plugin, you need to have at least one RGB color image open.

After clicking on Plugins ▶ Segmentation ▶ SIOX: Simple Interactive Object Extraction, the image will be embedded into the plugin GUI.

Step 1: Initial Segmentation.

Paint the regions of interest (ROIs) corresponding to the foreground and background. Select any of the ROI tools and mark the areas you consider foreground and background. To segment multiple objects, select "Allow multiple foreground components"

Step 2: Detail Refinement Brush.

Select new ROIs to be added or subtracted from the current segmentation. Press "Refine" to add/subtract those areas based on the previous result.

Step 3: Reset or Create mask.

Click the "Reset" button to restart the process or click on the "Create mask" button to create a binary image. Please, note that the binary image polarity will follow the option set in Process ▶ Binary ▶ Options.

SIOX segmentation Graphical User Interface applied to Leaf sample

Resulting mask of Leaf sample after SIOX segmentation

User Manual

SIOX is a plugin designed to segment 2D color images. If the image is not RGB color, it can be converted to RGB using the command Image ▶ Type ▶ RGB Color. However, the SIOX algorithm makes uses color information, so it is expected to work better on real RGB images.

When calling the plugin, the image will be embedded into the SIOX graphical user interface (GUI). This GUI has 3 section (panels): Initial Segmentation, Detail Refinement Brush and the Mask/Reset buttons.

Initial Segmentation

SIOX initial segmentation panel

This is the first step of the procedure where one defines the ROIs that correspond to typical areas of foreground or background.

Defining foreground and background

The radio buttons Foreground and Background switch between the two ROI types. When the foreground ROIs are being defined, the background ROIs are shown in translucent red. Similarly, when defining the background ROIs, the foreground ROIs are shown in translucent green.

The ROIs can be defined with any of the Selection Tools:

TIP: Multiple ROIs can be selected with any of the selection tools by pressing the SHIFT key and clicking on different parts of the image.

Multiple objects

For segmenting multiple objects, check Allow multiple foreground components. This option tells SIOX to look for several objects in the image. If you introduce multiple foreground ROIs, this option will be then assumed.

Smoothing

The Smoothing slider defines the sharpness of the resulting contours. For example, when segmenting the default sample "Leaf (36K)", reducing the smoothing provides more accurate borders.

Segmentation

Example of SIOX result after initial segmentation

In order to proceed with the initial segmentation, click on Segment.

Remember that SIOX needs at least one foreground component (ROI) to produce the segmentation, the background ROI is optional.

After a few seconds (depending on the image size), the result will be shown (darkened background and foreground area(s) in the original colors).

Following this, the initial segmentation panel is disabled. If the results are not satisfactory, press Reset to restart the process. Otherwise, continue to the next refinement step.

Detail Refinement Brush

At this point, the method can be called again to refine the results obtained so far to subtract or add new areas to the background or foreground components as many times needed.

The Add mode only modifies pixels formerly classified as background, while the Subtract mode only modifies those formerly classified as foreground.

Note: this step is optional. If you are already satisfied with the result, proceed to create the binary mask.

Clicking on subtract or add selects the ROIs to be added or subtracted. The sliders represent the threshold for the add and subtract refinement, deciding at which confidence level to stop at. Increasing the subtraction slider relaxes the threshold value to set an area as background, while decreasing the addition slider relaxes the threshold value to set an area as foreground.

Finally, by clicking on Refine, the segmentation is recalculated for the selected areas and the result will be displayed as before (darkened background and full color foreground). Repeat this step until you are completely satisfied.

Reset or Create mask

SIOX Reset/Create mask panel

This panel is enabled during any of the other segmentation steps.

Reset

Resets the image including the internal status of the confidence matrix (foreground, background and refinement areas). The initial ROIs are reloaded so they can be reused.

Create mask

Creates a binary (0-255) image based on the current state of the segmentation process. The background color (black or white) is defined in Process ▶ Binary ▶ Options... This button can be pressed during any of the previous steps.

Save segmentator

You can save the segmentation information into a file by clicking on the "Save segmentator" button. This can be used later to a different image or stack of images by clicking on Plugins ▶ Segmentation ▶ Apply SIOX segmentator (see next section).

Apply SIOX segmentator

You can apply a previously saved SIOX segmentator to any open image or set of images (stack).

The segmentator will be applied to the selected image or stack and the result will pop up.

Results of applying the previously calculated SIOX segmentator to a stack of transformed versions of the Leaf sample

Note: when applying a saved segmentator there is no information about the size of the expected foreground components so the largest component will be used as reference. The multiple component option will be consistent with what it was used during the segmentator calculation.