python - 使用 spaCy 3.0 将数据从旧的 Spacy v2 格式转换为全新的 Spacy v3 格式

Question

我有trainData具有以下简化格式的变量。

[

('Paragraph_A', {"entities": [(15, 26, 'DiseaseClass'), (443, 449, 'DiseaseClass'), (483, 496, 'DiseaseClass')]}),
('Paragraph_B', {"entities": [(969, 975, 'DiseaseClass'), (1257, 1271, 'SpecificDisease')]}),
('Paragraph_C', {"entities": [(0, 27, 'SpecificDisease')]})
]

我正在尝试trainData通过.spacy先将其转换doc为DocBin. 整个trainData文件可通过GoogleDocs访问。

我试图重现本教程中提到的内容，但对我不起作用。教程是：使用 spaCy 3.0 构建自定义 NER 模型

---

我尝试了以下。

import spacy
from spacy.tokens import DocBin

nlp = spacy.blank("en") # load a new spacy model
db = DocBin() # create a DocBin object

for text, annot in trainData: # data in previous format
    doc = nlp.make_doc(text) # create doc object from text
    ents = []
    for start, end, label in annot["entities"]: # add character indexes
        span = doc.char_span(start, end, label=label, alignment_mode="contract")
        ents.append(span)
    doc.ents = span # label the text with the ents
    db.add(doc)

db.to_disk("./train.spacy") # save the docbin object

但是我在如何将数据从 转换Spacy v2为Spacy v3. 在上面的代码片段中，我得到了一个回溯： TypeError: 'spacy.tokens.token.Token' object is not iterable.

标签： pythonnlpspacydata-conversion

Answer 1

你有一个小错误。检查 XXX 以获取更改的行。

import spacy
from spacy.tokens import DocBin

nlp = spacy.blank("en") # load a new spacy model
db = DocBin() # create a DocBin object

for text, annot in trainData: # data in previous format
    doc = nlp.make_doc(text) # create doc object from text
    ents = []
    for start, end, label in annot["entities"]: # add character indexes
        span = doc.char_span(start, end, label=label, alignment_mode="contract")
        ents.append(span)
    #XXX FOLLOWING LINE CHANGED
    doc.ents = ents # label the text with the ents
    db.add(doc)

db.to_disk("./train.spacy") # save the docbin object