时间戳

首页 > 解决方案 > 在特定的一个数据集上出现此错误

python - 在特定的一个数据集上出现此错误

问题描述

我正进入（状态

ValueError: `validation_steps=None` is only valid for a generator based on the `keras.utils.Sequence` class. Please specify `validation_steps` or use the `keras.utils.Sequence` class.

专门针对一个数据集（https://www.kaggle.com/dansbecker/5-celebrity-faces-dataset）。我实际上已经提到了验证步骤.fit_generator( )，它适用于我正在处理的所有其他数据集。这个有什么问题吗？

这是重要的代码片段（我正在使用 Keras）：

from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D, BatchNormalization
from keras.layers import Activation, Dropout, Flatten, Dense

training_data_dir = './train'
validation_data_dir = './validation'
MODEL_FILE='model.h5'

IMAGE_SIZE = 200
IMAGE_WIDTH, IMAGE_HEIGHT = IMAGE_SIZE, IMAGE_SIZE
EPOCHS = 100
BATCH_SIZE = 32
TEST_SIZE = 1
input_shape = (IMAGE_WIDTH, IMAGE_HEIGHT, 3)

training_data_generator = ImageDataGenerator(
    rescale=1./255,
    shear_range=0.1,
    zoom_range=0.1,
    horizontal_flip=True)
validation_data_generator = ImageDataGenerator(rescale=1./255)
test_data_generator = ImageDataGenerator(rescale=1./255)

training_generator = training_data_generator.flow_from_directory(
    training_data_dir,
    target_size=(IMAGE_WIDTH, IMAGE_HEIGHT),
    batch_size=BATCH_SIZE,
    class_mode="sparse")
validation_generator = validation_data_generator.flow_from_directory(
    validation_data_dir,
    target_size=(IMAGE_WIDTH, IMAGE_HEIGHT),
    batch_size=BATCH_SIZE,
    class_mode="sparse")

model = Sequential()

model.add(Conv2D(32, (3, 3), input_shape=input_shape, activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))

model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.25))

model.add(Conv2D(128, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))

model.add(Conv2D(256, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))

model.add(Flatten())

model.add(Dense(512, activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.3))

model.add(Dense(256, activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.4))

model.add(Dense(128, activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.5))

model.add(Dense(len(classes), activation='softmax'))

model.compile(loss='sparse_categorical_crossentropy',
            optimizer='nadam',
            metrics=['accuracy'])

model.fit_generator(
    training_generator,
    steps_per_epoch=len(training_generator.filenames) // BATCH_SIZE,
    epochs=EPOCHS,
    validation_data=validation_generator,
    validation_steps=len(validation_generator.filenames) // BATCH_SIZE, 
    verbose=1)
model.save_weights(MODEL_FILE)

标签： pythonpython-3.xtensorflowkeras

解决方案

validation_steps=len(validation_generator.filenames) // BATCH_SIZE 方法中的行model.fit_generator导致问题。因为，len(validation_generator.filenames)=25< BATCH_SIZE，32在上面的代码中，所以len(validation_generator.filenames) // BATCH_SIZE出来是0。

根据此处的文档，validation_steps通常应等于验证数据集的样本数除以批量大小。所以，让你的BATCH_SIZE小于或等于len(validation_generator.filenames)。

推荐阅读