Image Processing



  Basics   Image Operations
  Basic Operations   Erosion and Dilation
  OpenCV - Gaussian Blur
  OpevCV-Edge Detection
  Histogram Equalization   Histogram Plot
  Denoising Image
  BGR color palette
  Grabcut Algorithm
  Image Extraction   Video Reversing


Grabcut Algorithm

The syntax for grabCut() is: 

cv2.grabCut(img, mask, rect, bgdModel, fgdModel, iterCount[, mode])

Here are the descriptions on the parameters

  1. img : Input 8-bit 3-channel image.
  2. mask : Input/output 8-bit single-channel mask. The mask is initialized by the function when mode is set to GC_INIT_WITH_RECT. Its elements may have one of following values:
    1. GC_BGD defines an obvious background pixels.
    2. GC_FGD defines an obvious foreground (object) pixel.
    3. GC_PR_BGD defines a possible background pixel.
    4. GC_PR_FGD defines a possible foreground pixel.
  3. rect : ROI containing a segmented object. The pixels outside of the ROI are marked as obvious background. The parameter is only used when mode==GC_INIT_WITH_RECT .
  4. bgdModel : Temporary array for the background model. Do not modify it while you are processing the same image.
  5. fgdModel : Temporary arrays for the foreground model. Do not modify it while you are processing the same image.
  6. iterCount : Number of iterations the algorithm should make before returning the result. Note that the result can be refined with further calls with mode==GC_INIT_WITH_MASK or mode==GC_EVAL .
  7. mode : Operation mode that could be one of the following:
    1. GC_INIT_WITH_RECT The function initializes the state and the mask using the provided rectangle. After that it runs iterCount iterations of the algorithm.
    2. GC_INIT_WITH_MASK The function initializes the state using the provided mask. Note that GC_INIT_WITH_RECT and GC_INIT_WITH_MASK can be combined. Then, all the pixels outside of the ROI are automatically initialized with GC_BGD .
    3. GC_EVAL The value means that the algorithm should just resume.
Grab Cut Code:

We create a mask image similar to the loaded image:

mask = np.zeros(img.shape[:2],np.uint8)

Then, we create foregroundModel and backgroundModel. Then, we run the grabcut algorithm for 5 iterations with cv2.GC_INIT_WITH_RECT mode since we are using rectangle. It modifies the mask image:

cv2.grabCut(img,mask,rect,bgdModel,fgdModel,5,cv2.GC_INIT_WITH_RECT)

In this new mask image, pixels will be marked with four flags denoting background/foreground as specified above. So we modify the mask such that all 0-pixels and 2-pixels are put to 0 (ie background) and all 1-pixels and 3-pixels are put to 1(ie foreground pixels):

mask2 = np.where((mask==2)|(mask==0),0,1).astype('uint8')

Now our final mask is ready, and we can just multiply it with input image to get the segmented image:

img_cut = img*mask2[:,:,np.newaxis]

 

Example-1: Foreground Extraction in an Image using Grabcut Algorithm
#Grabcut Algorithm
import numpy as np 
import cv2 
from matplotlib import pyplot as plt 
   
image = cv2.imread('foreground.jpg') 
   
mask = np.zeros(image.shape[:2], np.uint8) 
   
backgroundModel = np.zeros((1, 65), np.float64) 
foregroundModel = np.zeros((1, 65), np.float64) 
   
rectangle = (20, 100, 150, 150) 
   
cv2.grabCut(image, mask, rectangle,   
            backgroundModel, foregroundModel, 
            3, cv2.GC_INIT_WITH_RECT) 
   
mask2 = np.where((mask == 2)|(mask == 0), 0, 1).astype('uint8') 
   
image = image * mask2[:, :, np.newaxis] 
   
plt.imshow(image) 
plt.colorbar() 
plt.show()
Output:
Input Image:
 
Output Image: