python - Title Extraction/Identification from PDFs

Question

I have a large number of pdfs in different formats. Among other things, I need to extract their titles (not the document name, but a title in the text). Due to the range of formats, the titles are not in the same locations in the pdfs. Further, some of the pdfs are actually scanned images (I need to use OCR/Optical Character Recognition on them). The titles are sometimes one line, sometimes 2. They do not tend to have the same set of words. In the range of physical locations the titles usually show up, there are often other words (ie if doc 1 has title 1 at x1, y1, doc 2 might have title 2 at x2, y2 but have other non-title text at x1 y1). Further, there are some very rare cases where the pdfs don't have a title.

So far I can use pdftotext to extract text within a given bounding box, and convert it to a text file. If there's a title, this lets me capture the title, but often with other extraneous words included. This also only works on non-image pdfs. I'm wondering if a) There's a good way to identify the title from among all the words I extract for a document (because there are often extraneous words), ideally with a good way to identify that no title exists, and b) if there are any tools that are equivalent to pdftotext that will also work on scanned images (I do have an ocr script working, but it does ocr over an entire image rather than a section of one).

One method that somewhat answers the title dilemma is to extract the words in the bounding box, use the rest of the document to identify which of the bounding box words are keywords for the document, and construct the title from the keywords. This wouldn't extract the actual title, but may give words that could construct a reasonable alternative. I'm already extracting keywords for other parts of the project, but I would definitely prefer to extract the actual title as people may be using the verbatim title for lookup purposes.

Further note if it wasn't clear - I'm trying to do this programatically with open source/free tools, ideally in Python, and I will have a large number of documents (10,000+).

标签： pythonpdfnlpocrpdf-scraping

Answer 1

您可以利用单词字体大小信息来提取标题单词。根据您的问题，我在这里理解的是我建议提取标题词的内容：

使用任何开源模块（例如pdf2image ）将 pdf 文档转换为图像，然后使用tesseract进行 OCR。从 OCR 输出中，您可以获得文本数据及其尺寸信息，即。单个单词的宽度和高度。

对单词的高度做一些统计分析（直方图），看看是否可以使用高度分布来识别标题词。您可以使用基于启发式信息的固定阈值，也可以使用基于高度分布的一些自适应阈值并使用此阈值来识别标题词。