时间戳

首页 > 解决方案 > 无限循环中的keras生成器

python - 无限循环中的keras生成器

问题描述

全部，

我对 Francois Chollet 的“使用 Python 进行深度学习”一书中的这段代码有疑问：

import numpy as np
import os
import sys
from matplotlib import pyplot as plt

def generator(data, lookback, delay, min_index, max_index,
shuffle=False, batch_size=128, step=6):

   if max_index is None:
      max_index = len(data) - delay - 1
   i = min_index + lookback
#   for it in range(1):
   while True:
      if shuffle:
         rows = np.random.randint(
              min_index + lookback, max_index, size=batch_size)
      else:
         if i + batch_size >= max_index:
              i = min_index + lookback
         rows = np.arange(i, min(i + batch_size, max_index))
         i += len(rows)
      samples = np.zeros((len(rows),
                   lookback // step,
                   data.shape[-1]))
      targets = np.zeros((len(rows),))
      for j, row in enumerate(rows):
        indices = range(rows[j] - lookback, rows[j], step)
        samples[j] = data[indices]
        targets[j] = data[rows[j] + delay][1]
   yield samples, targets

data_dir = os.path.join(os.getcwd(),'jena_climate')

fname = os.path.join(data_dir, 'jena_climate_2009_2016.csv')
f = open(fname)
data = f.read()
f.close()
lines = data.split('\n')
header = lines[0].split(',')
lines = lines[1:]
print(header)
print(len(lines))

float_data = np.zeros((len(lines), len(header) - 1))
for i, line in enumerate(lines):
       values = [float(x) for x in line.split(',')[1:]]
       float_data[i, :] = values

# temp = float_data[:, 1] 
# plt.plot(range(len(temp)), temp)
# plt.show()

mean = float_data[:200000].mean(axis=0)
float_data -= mean
std = float_data[:200000].std(axis=0)
float_data /= std

lookback = 1440
step = 6
delay = 144
batch_size = 128

train_gen = generator(float_data,
lookback=lookback,
delay=delay,
min_index=0,
max_index=200000,
shuffle=True,
step=step,
batch_size=batch_size)

val_gen = generator(float_data,
lookback=lookback,
delay=delay,
min_index=200001,
max_index=300000,
step=step,
batch_size=batch_size)

test_gen = generator(float_data,
lookback=lookback,
delay=delay,
min_index=300001,
max_index=None,
step=step,
batch_size=batch_size)

# samples,targets=next(train_gen)

val_steps = (300000 - 200001 - lookback)
test_steps = (len(float_data) - 300001 - lookback)

# print(val_steps)

def evaluate_naive_method():
    batch_maes = []
    for step in range(val_steps):
        samples, targets = next(val_gen) 
        print(step)
        preds = samples[:, -1, 1]
        mae = np.mean(np.abs(preds - targets))
        batch_maes.append(mae)

    print(np.mean(batch_maes))

evaluate_naive_method()

该代码定义了一个生成器函数来为温度预测示例生成样本。但是，在 Ubuntu linux 下安装我的 python 3.6.8 时，生成器仍然陷入无限循环，并且生成验证数据的循环永远不会执行。我在这里看到了类似的问题。显然，在 python 3 中正确处理生成器是一个小问题。有谁知道如何规避这个问题？

标签： pythonkerasgenerator

解决方案

是的，这是因为 yield 语句的缩进搞砸了。它应该包含在while True循环中，以便在循环的每次迭代中产生一个批次。改变：

while True:
    ### other code
yield samples, targets

至

while True:
    ### other code
    yield samples, targets

推荐阅读