python - 使用 river.stream.iter_csv() 循环数据集的 MemoryError

Question

https://www.pyimagesearch.com/2019/06/17/online-incremental-learning-with-keras-and-creme/ 我练习在线学习并点击此链接。

因为我错误地安装了 creme，所以我找到了替代工具 River。

当我运行块时：

dataset = stream.iter_csv('features.csv', target = 'class', converters=types)
model = Pipeline([
    ('scale', StandardScaler()),
    ('learn', OneVsRestClassifier(classifier=LogisticRegression()))
])

print('[INFO] model starting training...')
metric = Accuracy()
i = 1
for (X, y) in dataset:
    print('============================')
    preds = model.predict_one(X)
    model = model.fit_one(X, y)
    metric = metric.update(y, preds)
    print(f'[INFO] update {i}:{metric:5.2f}')
    i+=1
print(f'[INFO] final update: {metric}')

我看到内存错误：

[INFO] model starting training...
---------------------------------------------------------------------------
MemoryError                               Traceback (most recent call last)
<ipython-input-14-a18b62014885> in <module>
      2 metric = Accuracy()
      3 i = 1
----> 4 for (X, y) in dataset:
      5     print('============================')
      6     preds = model.predict_one(X)

c:\users\admin\appdata\local\programs\python\python37\lib\site-packages\river\stream\iter_csv.py in iter_csv(filepath_or_buffer, target, converters, parse_dates, drop, drop_nones, fraction, compression, seed, field_size_limit, **kwargs)
    147         buffer = utils.open_filepath(buffer, compression)
    148 
--> 149     for x in DictReader(fraction=fraction, rng=random.Random(seed), f=buffer, **kwargs):
    150 
    151         if drop:

c:\users\admin\appdata\local\programs\python\python37\lib\site-packages\river\stream\iter_csv.py in __next__(self)
     21 
     22         if self.line_num == 0:
---> 23             self.fieldnames
     24 
     25         row = next(self.reader)

c:\users\admin\appdata\local\programs\python\python37\lib\csv.py in fieldnames(self)
     96         if self._fieldnames is None:
     97             try:
---> 98                 self._fieldnames = next(self.reader)
     99             except StopIteration:
    100                 pass

MemoryError: 

任何人都知道为什么会发生此错误以及如何解决？

标签： pythononline-machine-learningcreme