Understanding Features of Image Segmentation Using OpenCV
In this article, We will learn to use marker-based Image Segmentation Using OpenCV with watershed algorithm.
Any grayscale image can be viewed as a topographic surface where high intensity denotes peaks and hills while low intensity denotes valleys. You start filling every isolated valley (local minima) with different colored water (labels). As the water rises, depending on the peaks (gradients) nearby, water from different valleys, obviously with different colors will start to merge.
To avoid that, you build barriers in the locations where water merges. You continue the work of filling water and building barriers until all the peaks are underwater. Then the barriers you created give you the segmentation result. This is the “philosophy” behind the watershed.
But this approach gives you over segmented results due to noise or any other irregularities in the image. So OpenCV implemented a marker-based watershed algorithm where you specify which are all valley points are to be merged and which are not.
Tutorial For Image Segmentation Using OpenCV
Below we will see an example on how to use the Distance Transform along with watershed to segment mutually touching objects.
Consider the coin’s image below, the coins are touching each other. Even if your threshold it, it will be touching each other.
We start by finding an approximate estimate of the coins. For that, we can use the Otsu’s binarization.
#import required library import numpy as np import cv2 as cv from matplotlib import pyplot as plt %matplotlib auto #Read the image using OpenCV. img = cv.imread("water_coins.jpg") cv.imshow('sample_image',img) cv.waitKey(0) # waits until a key is pressed cv.destroyAllWindows() # destroys the window showing image
gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY) ret,thresh=cv.threshold(gray,0,255,cv.THRESH_BINARY_INV+cv.THRESH_OTSU) cv.imshow('thresh_img',thresh) cv.waitKey(0) cv.destroyAllWindows()
Now we need to remove any small white noises in the image. For that, we can use a morphological opening. To remove any small holes in the object, we can use morphological closing. So, now we know for sure that region near to the center of objects is foreground and region much away from the object are backgrounds. The only region we are not sure of is the boundary region of coins.
So we need to extract the area in which we are sure they are coins. Erosion removes the boundary pixels. So whatever remaining, we can be sure it is a coin. That would work if objects were not touching each other. But since they are touching each other, another good option would be to find the distance transform and apply a proper threshold. Next, we need to find the area in which we are sure they are not coins. For that, we dilate the result.
Dilation increases object boundary to the background. This way, we can make sure whatever region in the background in the result is really a background since the boundary region is removed. See the image below.
Now, the remaining region is don’t know it is coins or background. with the watershed algorithm should find it. These areas are normally around the boundaries of coins where foreground and background meet (Or even two different coins meet). We call it a border. It can be obtained from subtracting sure_fg area from sure_bg area.
#noise removal kernel = np.ones((3,3), np.uint8) opening = cv.morphologyEx(thresh, cv.MORPH_OPEN, kernel, iterations=2) #sure background area sure_bg = cv.dilate(opening, kernel, iterations=3) #finding sure foreground area dist_transform=cv.distanceTransform(opening, cv.DIST_L2,5) ret,sure_fg=cv.threshold(dist_transform,0.7*dist_transform.max(),255,0) #finding unknown region sure_fg = np.uint8(sure_fg) unknown = cv.subtract(sure_fg, sure_bg) cv.imshow('unknown_img',sure_bg) cv.imshow('unknown_img',sure_fg) cv.waitKey(0) cv.destroyAllWindows()
See the output. In the thresholded image, we get some regions of coins that we are sure of coins and they are detached now.
Now we know for sure which are a region of coins, which are background and all. So we create a marker and label the regions inside it. The regions we know for sure (whether foreground or background) are labeled with any positive integers, but different integers and the area we don’t know for sure are just left as zero.
It labels the background of the image with 0, then other objects are labeled with integers starting from 1.
But if the background is marked with 0, watershed will consider it as unknown area. So we want to mark it with a different integer. Instead, we will mark the unknown region, defined by the unknown, with 0.
ret, markers = cv.connectedComponents(sure_fg)
#add one at all labels so that sure background is not 0, but 1
markers = markers+1
#now, mark the region of the unknown with 0
markers[unknown==255] = 0
See the output image, The dark blue region shows unknown region. Sure coins are colored with different values. The remaining area which is sure background are shown in lighter blue compared to unknown region.
It is time for final step, apply watershed. Then marker image will be modified. The boundary region will be marked with -1.
#apply watershed markers = cv.watershed(img, markers) img[markers == -1] = [255,0,0] cv.imshow('result_img',markers) cv.waitKey(0) cv.destroyAllWindows()
In this article we learned about Image Segmentation Using OpenCV, hence Concluded that, if any object is attached to each other and how it denote separate. So using a watershed algorithm in OpenCV and separate the object. Separation of object nothing but the boundary of every image known as ‘Image Segmentation’.