python - 使用 Python 自动裁剪图像以提取内部黑色边框 ROI

Question

我一直在研究 OpenCV 和 Pillow（以及 Python 之外的 ImageMagick，尤其是Fred 的 Image Magick Scripts）以实现以下目标：

自动识别扫描图像中的内部黑色边框并将图像裁剪到该边框。这是一个涂黑的示例图像，第一个是“原始”，第二个在黑色边框周围带有红色突出显示是我想要实现的：

问题是边界不在图像的外侧，扫描质量差异很大，这意味着边界永远不会在同一个位置，也无法通过像素进行裁剪。

编辑：我正在寻找一种方法来裁剪图像，只保留黑色边框内的所有内容（现在模糊的内容）

我正在寻求有关 a) 是否可以进行这种裁剪的帮助以及 b) 如何最好使用 Python 进行裁剪。

谢谢！

标签： pythonopencvimage-processingcomputer-visionimagemagick

Answer 1

这是在 Imagemagick 中执行此操作的一种非常简单的方法。

Get the center coordinates

Clone the image and do the following on the clone

Threshold the image so that the inside of the black lines is white. 

(If necessary use -connected-components to merge smaller black features into the white in the center)

Apply some morphology open to make sure that the black lines are continuous

Floodfill the image with red starting in the center

Convert non-red to black and red to white

Put the processed clone into the alpha channel of the input

输入：

center=$(convert img.jpg -format "%[fx:w/2],%[fx:h/2]\n" info:)

convert img.jpg \
\( +clone -auto-level -threshold 35% \
-morphology open disk:5 \
-fill red -draw "color $center floodfill" -alpha off \
-fill black +opaque red -fill white -opaque red \) \
-alpha off -compose copy_opacity -composite result.png

这是与上述等效的 Python Wand 代码：

#!/bin/python3.7

from wand.image import Image
from wand.drawing import Drawing
from wand.color import Color
from wand.display import display

with Image(filename='black_rect.jpg') as img:
    with img.clone() as copied:
        copied.auto_level()
        copied.threshold(threshold=0.35)
        copied.morphology(method='open', kernel='disk:5')
        centx=round(0.5*copied.width)
        centy=round(0.5*copied.height)
        with Drawing() as draw:
            draw.fill_color='red'
            draw.color(x=centx, y=centy, paint_method='floodfill')
            draw(copied)
        copied.opaque_paint(target='red', fill='black', fuzz=0.0, invert=True)
        copied.opaque_paint(target='red', fill='white', fuzz=0.0, invert=False)
        display(copied)
        copied.alpha_channel = 'copy'
        img.composite(copied, left=0, top=0, operator='copy_alpha')
        img.format='png'
        display(img)
        img.save(filename='black_rect_interior.png')

对于 OpenCV，我建议以下处理可能是一种方法。对不起，我不精通 OpenCV

Threshold the image so that the inside of the black lines is white. 

Apply some morphology open to make sure that the black lines are continuous

Get the contours of the white regions.

Get the largest interior contour and fill the inside with white

Put that result into the alpha channel of the input

添加：

对于那些有兴趣的人，这里有一个更长的方法，有利于透视校正。我做了一些类似于 nathancy 所做的事情，但在 Imagemagick 中。

首先，对图像进行阈值化并进行形态学打开以确保黑线是连续的。

然后做连通分量得到最大白色区域的ID号

然后提取该区域

id=$(convert img.jpg -auto-level -threshold 35% \
-morphology open disk:5 -type bilevel \
-define connected-components:mean-color=true \
-define connected-components:verbose=true \
-connected-components 8 null: | grep "gray(255)" | head -n 1 | awk '{print $1}' | sed 's/[:]*$//')
echo $id

convert img.jpg -auto-level -threshold 35% \
-morphology open disk:5 -type bilevel \
-define connected-components:mean-color=true \
-define connected-components:keep=$id \
-connected-components 8 \
-alpha extract -morphology erode disk:5 \
region.png

现在进行 Canny 边缘检测和霍夫线变换。这里我将canny图像，霍夫线保存为红线以及覆盖在图像上的线和线信息，保存在.mvg文件中。

convert region.png \
\( +clone -canny 0x1+10%+30% +write region_canny.png \
-background none -fill red -stroke red -strokewidth 2 \
-hough-lines 9x9+400 +write region_lines.png +write lines.mvg \) \
-compose over -composite region_hough.png

convert region_lines.png -alpha extract region_bw_lines.png

# Hough line transform: 9x9+400
viewbox 0 0 2000 2829
# x1,y1  x2,y2 # count angle distance
line 0,202.862 2000,272.704  # 763 92 824
line 204.881,0 106.09,2829  # 990 2 1156
line 1783.84,0 1685.05,2829  # 450 2 2734
line 0,2620.34 2000,2690.18  # 604 92 3240

接下来，我使用自己编写的脚本进行角点检测。这里我使用哈里斯检测器。

corners=$(corners -m harris -t 40 -d 5 -p yes region_bw_lines.png region_bw_lines_corners.png)
echo "$corners"

pt=1 coords=195.8,207.8
pt=2 coords=1772.8,262.8
pt=3 coords=111.5,2622.5
pt=4 coords=1688.5,2677.5

接下来，我以顺时针方式提取和排序角。以下是我写的一些我从这里转换的代码

list=$(echo "$corners" | sed -n 's/^.*=\(.*\)$/\1/p' | tr "\n" " " | sed 's/[ ]*$//' )
echo "$list"
195.8,207.8 1772.8,262.8 111.5,2622.5 1688.5,2677.5

# sort on x
xlist=`echo "$list" | tr " " "\n" | sort -n -t "," -k1,1`
leftmost=`echo "$xlist" | head -n 2`
rightmost=`echo "$xlist" | tail -n +3`
rightmost1=`echo "$rightmost" | head -n 1`
rightmost2=`echo "$rightmost" | tail -n +2`
# sort leftmost on y
leftmost2=`echo "$leftmost" | sort -n -t "," -k2,2`
topleft=`echo "$leftmost2" | head -n 1`
btmleft=`echo "$leftmost2" | tail -n +2`
# get distance from topleft to rightmost1 and rightmost2; largest is bottom right
topleftx=`echo "$topleft" | cut -d, -f1`
toplefty=`echo "$topleft" | cut -d, -f2`
rightmost1x=`echo "$rightmost1" | cut -d, -f1`
rightmost1y=`echo "$rightmost1" | cut -d, -f2`
rightmost2x=`echo "$rightmost2" | cut -d, -f1`
rightmost2y=`echo "$rightmost2" | cut -d, -f2`
dist1=`convert xc: -format "%[fx:hypot(($topleftx-$rightmost1x),($toplefty-$rightmost1y))]" info:`
dist2=`convert xc: -format "%[fx:hypot(($topleftx-$rightmost2x),($toplefty-$rightmost2y))]" info:`
test=`convert xc: -format "%[fx:$dist1>$dist2?1:0]" info:`
if [ $test -eq 1 ]; then
btmright=$rightmost1
topright=$rightmost2
else
btmright=$rightmost2
topright=$rightmost1
fi
sort_corners="$topleft $topright $btmright $btmleft"
echo $sort_corners

195.8,207.8 1772.8,262.8 1688.5,2677.5 111.5,2622.5

最后，我使用角坐标在黑色背景上绘制一个白色填充的多边形，并将结果放入输入图像的 alpha 通道。

convert img.jpg \
\( +clone -fill black -colorize 100 \
-fill white -draw "polygon $sort_corners" \) \
-alpha off -compose copy_opacity -composite result.png