python - 如何使用opencv检测手指数量并添加两位数?
问题描述
我是 OpenCV 和图像处理的新手。我正在尝试创建一个程序来检测相机前显示的手指数量并添加它们。在 GitHub 上搜索了一阵子,发现了一个检测手并数手指数量的项目。
我在执行过程中面临的问题是,除非我的背景很清楚,否则它不会向我显示数字。它不断变化的数字。我怎样才能稳定它?由于我需要将屏幕前显示的数字相加,除非它稳定,否则我无法存储该数字。
import traceback
import cv2
import numpy as np
import math
cap = cv2.VideoCapture(0)
while(1):
try: #an error comes if it does not find anything in window as it cannot find contour of max area
#therefore this try error statement
ret, frame = cap.read()
frame=cv2.flip(frame,1)
kernel = np.ones((3,3),np.uint8)
#define region of interest
roi=frame[100:300, 100:300]
cv2.rectangle(frame,(100,100),(300,300),(0,255,0),0)
hsv = cv2.cvtColor(roi, cv2.COLOR_BGR2HSV)
# define range of skin color in HSV
lower_skin = np.array([0,20,70], dtype=np.uint8)
upper_skin = np.array([20,255,255], dtype=np.uint8)
#extract skin colur imagw
mask = cv2.inRange(hsv, lower_skin, upper_skin)
#extrapolate the hand to fill dark spots within
mask = cv2.dilate(mask,kernel,iterations = 4)
#blur the image
mask = cv2.GaussianBlur(mask,(5,5),100)
#find contours
contours,hierarchy= cv2.findContours(mask,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
print(contours)
print(hierarchy)
#find contour of max area(hand)
cnt = max(contours, key = lambda x: cv2.contourArea(x))
#approx the contour a little
epsilon = 0.0005*cv2.arcLength(cnt,True)
approx= cv2.approxPolyDP(cnt,epsilon,True)
#make convex hull around hand
hull = cv2.convexHull(cnt)
#define area of hull and area of hand
areahull = cv2.contourArea(hull)
areacnt = cv2.contourArea(cnt)
#find the percentage of area not covered by hand in convex hull
arearatio=((areahull-areacnt)/areacnt)*100
#find the defects in convex hull with respect to hand
hull = cv2.convexHull(approx, returnPoints=False)
defects = cv2.convexityDefects(approx, hull)
# l = no. of defects
l=0
#code for finding no. of defects due to fingers
for i in range(defects.shape[0]):
s,e,f,d = defects[i,0]
start = tuple(approx[s][0])
end = tuple(approx[e][0])
far = tuple(approx[f][0])
pt= (100,180)
# find length of all sides of triangle
a = math.sqrt((end[0] - start[0])**2 + (end[1] - start[1])**2)
b = math.sqrt((far[0] - start[0])**2 + (far[1] - start[1])**2)
c = math.sqrt((end[0] - far[0])**2 + (end[1] - far[1])**2)
s = (a+b+c)/2
ar = math.sqrt(s*(s-a)*(s-b)*(s-c))
#distance between point and convex hull
d=(2*ar)/a
# apply cosine rule here
angle = math.acos((b**2 + c**2 - a**2)/(2*b*c)) * 57
# ignore angles > 90 and ignore points very close to convex hull(they generally come due to noise)
if angle <= 90 and d>30:
l += 1
cv2.circle(roi, far, 3, [255,0,0], -1)
#draw lines around hand
cv2.line(roi,start, end, [0,255,0], 2)
l+=1
#print corresponding gestures which are in their ranges
font = cv2.FONT_HERSHEY_SIMPLEX
if l==1:
if areacnt<2000:
cv2.putText(frame,'Put hand in the box',(0,50), font, 2, (0,0,255), 3, cv2.LINE_AA)
else:
if arearatio<12:
cv2.putText(frame,'0',(0,50), font, 2, (0,0,255), 3, cv2.LINE_AA)
elif arearatio<17.5:
cv2.putText(frame,'Best of luck',(0,50), font, 2, (0,0,255), 3, cv2.LINE_AA)
else:
cv2.putText(frame,'1',(0,50), font, 2, (0,0,255), 3, cv2.LINE_AA)
elif l==2:
cv2.putText(frame,'2',(0,50), font, 2, (0,0,255), 3, cv2.LINE_AA)
elif l==3:
if arearatio<27:
cv2.putText(frame,'3',(0,50), font, 2, (0,0,255), 3, cv2.LINE_AA)
else:
cv2.putText(frame,'ok',(0,50), font, 2, (0,0,255), 3, cv2.LINE_AA)
elif l==4:
cv2.putText(frame,'4',(0,50), font, 2, (0,0,255), 3, cv2.LINE_AA)
elif l==5:
cv2.putText(frame,'5',(0,50), font, 2, (0,0,255), 3, cv2.LINE_AA)
elif l==6:
cv2.putText(frame,'reposition',(0,50), font, 2, (0,0,255), 3, cv2.LINE_AA)
else :
cv2.putText(frame,'reposition',(10,50), font, 2, (0,0,255), 3, cv2.LINE_AA)
#show the windows
cv2.imshow('mask',mask)
cv2.imshow('frame',frame)
except Exception:
traceback.print_exc()
pass
# break
k = cv2.waitKey(5) & 0xFF
if k == 27:
break
cv2.destroyAllWindows()
cap.release()
这是完整的代码。我觉得肤色的 HSV 范围不准确。我是一个棕色的印度人,在遮罩屏幕中,它使皮肤范围颜色为白色,其余为 1,我可以看到我手掌中的某些区域是黑色的。
有没有办法通过保持时间限制来添加两位数?例如,当我显示 5 并且在 1 秒内,如果我显示 3,它应该将输入作为 53。
带有一些示例或演示的答案将有很大帮助。谢谢你。
解决方案
从数据中消除抖动的最常用方法是收集更多数据并从中获取平均或最常出现的数据。
您可以测量一秒钟,然后取其中出现最多的数字。例如,如果您有 10 个帧分别显示 [5, 3, 1, 5, 5, 4, 5, 4, 3, 2] 个手指,则输出应该是 5 个手指。然后,您将手指数量添加到之前测量的手指数量中,将其显示在屏幕上并继续测量。
首先,我认为制作一种拍摄图像并返回测量的手指数量的方法是个好主意。让我们称之为DetectNrFingers(图像)。您应该能够从上面的所有代码中轻松获得这一点,因为您知道哪个变量打印手指的数量。
您想添加两位数。这是一个额外的问题。如果你总是有两位数,那很容易,你可以总共测量 4 次,例如,5,3,4,2。那么答案就是 53 + 42。如果不是一直是两位数,你可以让对方在一位数前先举起 0,然后仍然测量 4 次。
我通常使用 C++ 工作,所以制作代码示例可能充满错误。我用一种 python 风格做了一些伪代码,所以希望你能遵循它。所以让我们伪代码:
//set the start time so you know how much time has passed after the first measurement
start_time = time.time()
//this is how long you want to measure the fingers
timeToMeasure = 1000
//initialize with a not possible number
previousAmountFingers = -1
prevCombinedFingers = -1
while(1):
//Get the image Mat from the camera. It can be the raw feed, or it can be preprocessed depending on how you want to display it later.
image = GetImageFromCamera()
listNrOfFingers.append(DetectNrFingers(image))
elapsed_time = time.time() - start_time
if(elapsed_time > timeToMeasure)
//this is my example above, which would return a 5 in [5, 3, 1, 5, 5, 4, 5, 4, 3, 2]
amountOfFingers = numberOccuringTheMost(listNrOfFingers)
listNrOfFingers.Clear()
if(previousAmountFingers != -1) //the initial value
//combine seperate integers to one, for example, 5 and 3 -> 53
combinedFingers = int(str(amountOfFingers) + str(previousAmountFingers))
previousAmountFingers = -1
if(prevCombinedFingers != -1)
sumFingers = combinedFingers + prevCombinedFingers
//putText sumFingers into the image on a position you like
prevCombinedFingers = combinedFingers
else //only if the previous digit was not initialized yet:
previousAmountFingers = amountOfFingers
start_time = time.time() //reset the timer for the next round of measurements.
由于您需要实时执行,我认为每帧 1 秒是不够的,但是我猜您可以稍微延迟计算。只需打印先前计算的总和,直到计算下一个总和。因此,即使您尚未计算手指,也请多显示帧。当然你可以将 1 秒设置得更短,但你也需要时间来调整你的手,所以如果你想展示手本身,无论如何你需要展示比答案更多的帧。
推荐阅读
- java - Spring WebFlux Mono 的异常中断 Mono.zip
- python - 为什么 python 的 SharedMemory 似乎将数组初始化为零
- power-automate - Power Automate“发送电子邮件 (V2) 似乎不支持 UTF-8 字符,有什么办法吗?
- sql - Option(Optimize For Unknown) 和 option(Optimize for (@parameter Unknown)) 之间的区别
- javascript - 如何在更换新护照之前检查密码是否匹配
- javascript - 为什么 React 不能正确渲染新创建的帖子?
- c++ - C++:GCC 中 localtime_s 的位置
- flutter - 多级下拉按钮 Flutter - 州和城市
- stm32 - 不能用 LDR 替换 MOVW / MOVT
- excel - 有没有办法将格式相似的 Excel 选项卡中的所有数据合并到同一工作簿中的单独选项卡中并能够同步编辑?