时间戳

首页 > 解决方案 > fastai: ValueError: __len__() 应该返回 >= 0

deep-learning - fastai: ValueError: __len__() 应该返回 >= 0

问题描述

运行以下程序时 - https://rawgit.com/sizhky/eef1482e63387df8e9e045ac1e5a0ce8/raw/bdbebafaab21739a27f6bf32e83da1557919b44b/lm.html

我无法打电话learner.fit，因为它会引发上述错误。

具体来说，
我正在尝试训练一个语言模型，获取一个文本文件并将其转换为 aLanguageModelData并通过以下方式将其提供给 RNNget_model

md = LanguageModelData.from_text_files(PATH, TEXT, **FILES, bs=bs, bptt=bptt, min_freq=10)
learner = md.get_model(opt_fn, em_sz, nh, nl, dropouti=0.05, dropout=0.05, wdrop=0.1, dropoute=0.02, dropouth=0.05) learner.reg_fn = partial(seq2seq_reg, alpha=2, beta=1) learner.clip=0.3 learner.fit(3e-3, 4) 

ValueErrorTraceback (most recent call last)
<ipython-input-7-579772ee6693> in <module>()
----> 1 learner.fit(3e-3, 4)

/nfsroot/data/home/yeshwanth/misc/fastai/fastai/courses/dl1/practice/fastai/learner.py in fit(self, lrs, n_cycle, wds, **kwargs)
    285         self.sched = None
    286         layer_opt = self.get_layer_opt(lrs, wds)
--> 287         return self.fit_gen(self.model, self.data, layer_opt, n_cycle, **kwargs)
    288 
    289     def warm_up(self, lr, wds=None):

/nfsroot/data/home/yeshwanth/misc/fastai/fastai/courses/dl1/practice/fastai/learner.py in fit_gen(self, model, data, layer_opt, n_cycle, cycle_len, cycle_mult, cycle_save_name, best_save_name, use_clr, use_clr_beta, metrics, callbacks, use_wd_sched, norm_wds, wds_sched_mult, use_swa, swa_start, swa_eval_freq, **kwargs)
    232             metrics=metrics, callbacks=callbacks, reg_fn=self.reg_fn, clip=self.clip, fp16=self.fp16,
    233             swa_model=self.swa_model if use_swa else None, swa_start=swa_start,
--> 234             swa_eval_freq=swa_eval_freq, **kwargs)
    235 
    236     def get_layer_groups(self): return self.models.get_layer_groups()

/nfsroot/data/home/yeshwanth/misc/fastai/fastai/courses/dl1/practice/fastai/model.py in fit(model, data, n_epochs, opt, crit, metrics, callbacks, stepper, swa_model, swa_start, swa_eval_freq, **kwargs)
    159 
    160         if not all_val:
--> 161             vals = validate(model_stepper, cur_data.val_dl, metrics, seq_first=seq_first)
    162             stop=False
    163             for cb in callbacks: stop = stop or cb.on_epoch_end(vals)

/nfsroot/data/home/yeshwanth/misc/fastai/fastai/courses/dl1/practice/fastai/model.py in validate(stepper, dl, metrics, seq_first)
    220     stepper.reset(False)
    221     with no_grad_context():
--> 222         for (*x,y) in iter(dl):
    223             y = VV(y)
    224             preds, l = stepper.evaluate(VV(x), y)

/nfsroot/data/home/yeshwanth/misc/fastai/fastai/courses/dl1/practice/fastai/nlp.py in __next__(self)
    135 
    136     def __next__(self):
--> 137         if self.i >= self.n-1 or self.iter>=len(self): raise StopIteration
    138         bptt = self.bptt if np.random.random() < 0.95 else self.bptt / 2.
    139         seq_len = max(5, int(np.random.normal(bptt, 5)))

ValueError: __len__() should return >= 0

标签： deep-learninglanguage-model

解决方案

看起来您的数据位于 1 个 .txt 文件中，并LanguageModelData.from_text_files()希望处理包含许多文件的文件夹

UPD：解决了！每个文件夹中必须至少有bs多个文件！否则LanguageModelLoaderforLanguageModelData的数据为空。

我在验证过程中遇到了同样的错误，问题似乎出在LanguageModelData()构建数据集的方式上：

for (*x, y) in md.trn_dl:
    set_trace()

x应该是 pytorch 张量或 shape (smth, batch_size)，以及y- 大小为 1 维的张量smth*batch_size。与 相同md.val_dl。在您的情况下，很可能没有(*x, y). 数据有问题。len(md.trn_dl)并且len(md.val_dl)不能等于 0。

我将不胜感激任何解决方案，谢谢您的提问！

此外，新版本的语言模型下降torchtext并更容易调试：https ://github.com/fastai/fastai/blob/master/courses/dl2/imdb.ipynb

推荐阅读