AI: Real World Challenges with Image Segmentation

While several toy applications exist today that illustrate the basic capabilities of image classification, the field of accurate Image Segmentation poses several key challenges in real world applications. The case of importance is to be able to effectively segment medical images. The purpose of image segmentation is to partition an image into mutually exclusive regions, each of which has homogeneous properties that are significantly different from those of the neighboring regions. Sounds simple, lets delve a little deeper into our learnings from designing real world applications for Medical Image Segmentation – case in point, segmenting human brain! We chose brain to illustrate key learnings from the challenges that emanate with progressive complexity -ability to distinguish soft tissue for instance is a key challenge due to overlapping regions due to noise from MRI scans.

Why is Image Segmentation important? In clinical applications, image segmentation can aid in measuring and visualizing the brain’s anatomical structures, for analyzing brain changes, for delineating pathological regions, and for surgical planning and image-guided interventions. Accurate tissue segmentation can be used to obtain quantitative results of physiological change, for instance, in brain tumor imaging, brain tumor segmentation can be used to identify the size and location of the tumor. The tumor state and progression of tumor growth can aid in radiation planning (maximizing the probability of eradication of tumor tissue and minimize the damage caused to good tissue). This approach of using Image Segmentation can also inform effectiveness of drugs.

Image Segmentation Approach

MRI is one of the most commonly utilized medical imaging techniques. It provides good contrast for dissimilar tissues and offers predominance over CT and other diagnostic imaging techniques for brain tissue studies, making it an attractive method for most image segmentation & feature extraction. As an increasing number of medical images are produced, automatic image processing and analysis techniques have become essential.

Development of brain image segmentation has numerous applications – segmenting into Cerebrospinal fluid (CSF), gray matter (GM) and white matter (WM) are the principal components of the brain and they are the main targets for image segmentation in brain MR images. Every image consists of a finite set of image elements called pixels in 2D space or voxels in 3D space. An image can be defined as a function I(i,j) in 2D space or I(i,j,k) in 3D space, where i,j,k denotes special denote spatial coordinates. The values (or amplitudes) of the functions are intensity values and are typically represented by a gray value {0,..,255} in MRI of the brain.

MRI brain images from Brainweb was used for experimenting and fine-tuning models. Brainweb is a simulated brain MRI database that includes a set of realistic MRI data volumes produced by an MRI simulator – the “ground truth” of each tissue is obtained from MRI volumes under different types of conditions, such as noise level, slice thickness and so on. We utilize this data to estimate the performance of diverse image segmentation methods compared to the known “ground truth”. Three orthogonal views (transversal, sagittal, and coronal) were used for the purpose of this study. The goal was to segment the images into four classes: background, CSF, GM and WM.

Challenges with Brain Image Segmentation

1. Separating Brain tissue from the Non-Brain: The first challenge in segmentation is to separate the “brain” matter from “non-brain” (like skull). The intensity of brain tissue is one of the most important features for brain MRI segmentation. However, we observed when intensity values are corrupted by MRI artifacts such as image noise, partial volume effect (PVE), and bias field effect, intensity-based segmentation algorithms will lead to wrong results.

2. To improve image quality, one approach is to consider increasing scan time. Greater the image scan time from an MRI, the greater the resolution. Well, in practice, adult brain MRI studies image acquisition time is around 20 min, which affects special resolution. Greater spatial resolution can be obtained with a longer scanning time, but this must be weighed against patient exposure from radiation and discomfort.

3. Image segmentation based on individual pixel/voxel intensities (first order features) is feasible only when intensities of an object of interest and its background differ to a large extent. At the outset, Fuzzy C-Means (FCM) algorithm seems like a good candidate – FCM is based on minimizing an object function by iteratively updating the membership function and cluster centers. This approach is prone to work well in noise free images, but the accuracy of segmentation is inversely proportional to noise within the medical image. This is because the FCM algorithm only utilizes the gray level information of each pixel and ignores spatial contextual information.

4. To compensate for noise reduction, since noise is independent of location, spatial constraints can reduce noise disturbance. However, the smoothing effect of the spatial information can lose details in the original image when it suppresses noise and the overall fidelity of image will be lost to filter noise.

Solution: To obtain relevant and accurate segmentation results, very often several preprocessing steps are necessary to prepare MRI data.

For instance, it is necessary to remove background voxels, extract brain tissue, and separate Nonbrain tissues such as fat, skull, or neck have intensities overlapping with intensities of brain tissues. When intensity values are corrupted by MRI artifacts such as image noise, partial volume effect (PVE), and bias field effect, intensity-based segmentation algorithms will lead to wrong results. Therefore, rational and proper utilization of spatial information for image segmentation is critical.>

Therefore, the brain must be extracted before brain segmentation methods can be used. This step classifies voxels as brain or nonbrain. The result can be either a new image with just brain voxels or a binary mask, which has a value of 1 for brain voxels and 0 for the rest of tissues as shown:

From a modeling perspective, from a given brain slice, we innovatively combine FCM (that utilize luminance information from images) with Markov random field (MRF) to model spatial or contextual dependencies (to procure neighborhood context) in images. FCM is good to analyze and utilize luminance information from images and MRF can facilitate modeling of the spatial or contextual dependencies without compromising on fidelity of image. In addition, we leverage optimization techniques such as Ant colony optimization and a Gossiping algorithm, for segmenting MRIs in real time environments.

Our original approach of developing a single algorithm to segment the brain into white matter, gray matter and cerebrospinal fluid – but, results weren’t convincing. This was because all three layers had different characteristics – the white matter consisted of large and broad white patches, the cerebrospinal fluid was thinner and wiry, and the gray matter consisted of both characteristics. Thus, we developed one segmentation algorithm for each layer with separate denoisers using Autoencoders. An autoencoder is a neural network which is used for dimensionality reduction, as well as feature extraction and selection.

The resulting approach yields the following segmentation, described in the diagram below:

The approach above is compute intensive. Now, we assume infinite power of cloud to train the model – however, the question is how to we deliver advanced intelligence to edge nodes, that have processing power of a tablet at-most – all in real-time? Drop us a note! Stay tuned for our insights in our next blog post! Carpe Diem’– R

Leave a Comment