NMS (Non-Maximum Suppression)

1. Non-Maximum Suppression for Object Detection in Python - PyImageSearch

Non-Maximum Suppression for Object Detection in Python - PyImageSearch

2. (Faster) Non-Maximum Suppression in Python - PyImageSearch

(Faster) Non-Maximum Suppression in Python - PyImageSearch

3. Non Max Suppression Explained and PyTorch Implementation - YouTube

---

왜 NMS를 사용해야 하는가?

만약 이미지 안에 객체가 하나 있는데 여러 개의 박스가 그려지면 큰 문제이다. 예를 들어 오드리 햅번의 얼굴을 검출하는 객체 탐지를 한다고 했을 때, 결과물로 6개의 박스를 그려내면서 6개의 박스가 있다고 하면 정말 끔찍할 것이다! 그래서 가장 좋은 바운딩 박스(혹은 proposals)를 제외하고(Non-Maximum) 나머지는 억제(suppress)해주는 과정이 필요하다.

오드리 햅번과 박스들

---

nms.py라는 파일을 하나 만들자.

nms.py

먼저 라인 2에서 numpy 패키지를 임포트해준다.

다음으로, non_max_suppression_slow라는 함수를 라인 5에 정의해준다. 이 함수는 두 개의 인자를 받아들인다: (1) (startX, startY, endX, endY)의 형태를 갖는 바운딩 박스, (2) 중첩 임계치(overlap threshold). 두번째 인자에 대해서는 추후 논의하도록 하겠다.

라인 7-8d에선 바운딩 박스에 대한 간단한 검사를 한다. 만약 바운딩 박스가 리스트 안에 없다면 빈 리스트를 함수 호출자에게 반환한다.

이후, 우리가 남기고 싶은 바운딩 박스를 담기 위한 리스트 변수 pick을 초기화 한다 (11).

gpripe.jpg

raccoon-1.zip

0.06MB

# import the necessary packages
from pyimagesearch.nms import non_max_suppression_slow
import numpy as np
import cv2

# construct a list containing the images that will be examined
# along with their respective bounding boxes
images = [
	("images/raccoon-1.jpg", np.array([
	(81, 88, 522, 408),
    (93, 88, 534, 408),
    (105, 88, 546, 408),
    (81, 100, 522, 420),
    (93, 100, 534, 420),
    (93, 112, 534, 436)]))
]
    
# loop over the images
for (imagePath, boundingBoxes) in images:
	# load the image and clone it
	print "[x] %d initial bounding boxes" % (len(boundingBoxes))
	image = cv2.imread(imagePath)
	orig = image.copy()
    
	# loop over the bounding boxes for each image and draw them
	for (startX, startY, endX, endY) in boundingBoxes:
		cv2.rectangle(orig, (startX, startY), (endX, endY), (0, 0, 255), 2)
        
	# perform non-maximum suppression on the bounding boxes
	pick = non_max_suppression_slow(boundingBoxes, 0.3)
	print "[x] after applying non-maximum, %d bounding boxes" % (len(pick))
    
	# loop over the picked bounding boxes and draw them
	for (startX, startY, endX, endY) in pick:
		cv2.rectangle(image, (startX, startY), (endX, endY), (0, 255, 0), 2)
        
	# display the images
	cv2.imshow("Original", orig)
	cv2.imshow("After NMS", image)
	cv2.waitKey(0)

# nms_slow.py

# import the necessary packages
import numpy as np

#  Felzenszwalb et al.
def non_max_suppression_slow(boxes, overlapThresh):
	# if there are no boxes, return an empty list
	if len(boxes) == 0:
		return []
        
	# initialize the list of picked indexes
	pick = []
    
	# grab the coordinates of the bounding boxes
	x1 = boxes[:,0]
	y1 = boxes[:,1]
	x2 = boxes[:,2]
	y2 = boxes[:,3]
    
	# compute the area of the bounding boxes and sort the bounding
	# boxes by the bottom-right y-coordinate of the bounding box
	area = (x2 - x1 + 1) * (y2 - y1 + 1)
	idxs = np.argsort(y2)
    
	# keep looping while some indexes still remain in the indexes
	# list
	while len(idxs) > 0:
		# grab the last index in the indexes list, add the index
		# value to the list of picked indexes, then initialize
		# the suppression list (i.e. indexes that will be deleted)
		# using the last index
		last = len(idxs) - 1
		i = idxs[last]
		pick.append(i)
		suppress = [last]
        
		# loop over all indexes in the indexes list
		for pos in xrange(0, last):
			# grab the current index
			j = idxs[pos]
            
			# find the largest (x, y) coordinates for the start of
			# the bounding box and the smallest (x, y) coordinates
			# for the end of the bounding box
			xx1 = max(x1[i], x1[j])
			yy1 = max(y1[i], y1[j])
			xx2 = min(x2[i], x2[j])
			yy2 = min(y2[i], y2[j])
            
			# compute the width and height of the bounding box
			w = max(0, xx2 - xx1 + 1)
			h = max(0, yy2 - yy1 + 1)
            
			# compute the ratio of overlap between the computed
			# bounding box and the bounding box in the area list
			overlap = float(w * h) / area[j]
            
			# if there is sufficient overlap, suppress the
			# current bounding box
			if overlap > overlapThresh:
				suppress.append(pos)
                
		# delete all indexes from the index list that are in the
		# suppression list
		idxs = np.delete(idxs, suppress)
        
	# return only the bounding boxes that were picked
	return boxes[pick]

 

---

YouTube PyTorch Implementation

src github: Machine-Learning-Collection/nms.py at master · aladdinpersson/Machine-Learning-Collection (github.com)

import torch
from iou import intersection_over_union

def nms(bboxes, iou_threshold, threshold, box_format="corners"):
    """
    Does Non Max Suppression given bboxes
    Parameters:
        bboxes (list): list of lists containing all bboxes with each bboxes
        specified as [class_pred, prob_score, x1, y1, x2, y2]
        iou_threshold (float): threshold where predicted bboxes is correct
        threshold (float): threshold to remove predicted bboxes (independent of IoU) 
        box_format (str): "midpoint" or "corners" used to specify bboxes
    Returns:
        list: bboxes after performing NMS given a specific IoU threshold
    """

    assert type(bboxes) == list

    bboxes = [box for box in bboxes if box[1] > threshold]
    bboxes = sorted(bboxes, key=lambda x: x[1], reverse=True)
    bboxes_after_nms = []

    while bboxes:
        chosen_box = bboxes.pop(0)

        bboxes = [
            box
            for box in bboxes
            if box[0] != chosen_box[0]
            or intersection_over_union(
                torch.tensor(chosen_box[2:]),
                torch.tensor(box[2:]),
                box_format=box_format,
            )
            < iou_threshold
        ]

        bboxes_after_nms.append(chosen_box)

    return bboxes_after_nms