时间戳

首页 > 解决方案 > How to train an NER model with different beam objective parameters in Spacy?

python - How to train an NER model with different beam objective parameters in Spacy?

问题描述

I'm attempting to update a pre-trained spacy model en_core_web_md with a few rounds of a beam objective other than beam_width = 1, and I cannot seem to find the right way to pass the different parameters into the **cfg such that the model uses them for training (at THIS point).

This was my latest attempt:

pipe_exceptions = ["ner", "trf_wordpiecer", "trf_tok2vec"]
other_pipes = [pipe for pipe in nlp.pipe_names if pipe not in pipe_exceptions]
# only train NER
with nlp.disable_pipes(*other_pipes), warnings.catch_warnings():
    # show warnings for misaligned entity spans once
    warnings.filterwarnings("once", category=UserWarning, module='spacy')

    # TRY TO FORCE BEAM TRAINING INSTEAD OF GREEDY METHOD
    nlp.use_params({'ner':{'beam_width':16, 'beam_density':0.0001}})
    print(nlp.meta) 

    sizes = compounding(1.0, 4.0, 1.001)
    # batch up the examples using spaCy's minibatch
    for itn in range(n_iter):
        random.shuffle(TRAIN_DATA_2)
        batches = minibatch(TRAIN_DATA_2, size=sizes)
        losses = {}
        for batch in batches:
            texts, annotations = zip(*batch)
            nlp.update(texts, 
            annotations, 
            sgd=optimizer, 
            drop=0.35, 
            losses=losses
            )
        print("Losses", losses)

However, after training, the model/ner/cfg file still lists:

{
"beam_width":1,
"beam_density":0.0,
"beam_update_prob":1.0,
...

So, I have a few questions:

  1. Am I able to update an existing greedy trained model with a new beam objective?
  2. If true, how can I properly changed these training parameters (and confirm they are changed)?
  3. If false, for a new from scratch model, how can I properly changed these training parameters (and confirm they are changed)?

Why do this? I am attempting to train a model that provides probabilities for NER decisions that I can surface to my users. THIS post and a few others show how to use beam_parse to obtain probabilities after the fact from a greedy model. However, they all mention that the greedy model hasn't been trained with a global objective, so these scores aren't especially meaningful unless you also perform some iterations of beam training as well. (link to github issue)

标签： pythonspacy

解决方案

我在这个堆栈帖子中找到了答案。这是修改配置参数的语法。

nlp.entity.cfg['beam_width'] = 16
nlp.entity.cfg['beam_density'] = 0.0001

我把这些行放在前面optimizer = nlp.resume_training()，这些值用于训练。

推荐阅读