python - Keras图像分类模型的增量训练
问题描述
我使用了较小的 VGG 模型并修改了以下教程的训练脚本来训练之前训练的模型。模型和脚本的原始来源: https ://www.pyimagesearch.com/2018/04/16/keras-and-convolutional-neural-networks-cnns/
这就是我所做的:
第一堂训练课:
- 使用教程中的原始训练脚本使用 2 类 A 和 B 的图像数据集训练模型
第二期培训:
- 加载训练好的 keras 模型并使用 C 类的图像数据集进行训练,没有任何 A 类和 B 类的新数据,使用以下修改后的训练脚本(加载和保存模型方法引用自以下 stackoverflow 线程:Loading a train a trained Keras model and continue培训)
- 加载第一个会话的腌制标签数组,将其与第二个会话中的新标签数组结合起来,并将其保存在 lb.pickle
结果:
2nd session 之后的训练模型只能识别 2nd session 中的新类。似乎在第一节训练的其他课程丢失了。它只是行不通。
我的问题: 如何修复以下脚本以进行增量培训?或者与我的案例类似的增量培训的任何其他建议或参考?
我修改后的训练脚本:
from keras.utils import np_utils
from keras.preprocessing.image import ImageDataGenerator
from keras.optimizers import Adam
from keras.preprocessing.image import img_to_array
from keras.models import load_model
from sklearn.preprocessing import LabelBinarizer
from sklearn.model_selection import train_test_split
from smallervggnet import SmallerVGGNet
from imutils import paths
import numpy as np
import argparse, os, sys
import random
import pickle
import cv2
ap = argparse.ArgumentParser()
ap.add_argument("-d", "--dataset", required=True,
help="path to input dataset (i.e., directory of images)")
ap.add_argument("-im", "--loadmodel", required=True,
help="path to model to be loaded")
ap.add_argument("-m", "--model", required=True,
help="path to output model")
ap.add_argument("-l", "--labelbin", required=True,
help="path to output label binarizer")
ap.add_argument("-p", "--plot", type=str, default="plot.png",
help="path to output accuracy/loss plot")
args = vars(ap.parse_args())
EPOCHS = 100
INIT_LR = 1e-3
BS = 10
IMAGE_DIMS = (256, 256, 3)
data = []
labels = []
print("[INFO] loading images...")
imagePaths = sorted(list(paths.list_images(args["dataset"])))
random.seed(42)
random.shuffle(imagePaths)
for imagePath in imagePaths:
image = cv2.imread(imagePath)
image = cv2.resize(image, (IMAGE_DIMS[1], IMAGE_DIMS[0]))
image = img_to_array(image)
data.append(image)
label = imagePath.split(os.path.sep)[-2]
labels.append(label)
data = np.array(data, dtype="float") / 255.0
labels = np.array(labels)
print("[INFO] data matrix: {:.2f}MB".format(
data.nbytes / (1024 * 1000.0)))
lb = LabelBinarizer()
bLabels = lb.fit_transform(labels)
(trainX, testX, trainY, testY) = train_test_split(data,
bLabels, test_size=0.2, random_state=42)
#add these 2 lines to avoid error
trainY = np_utils.to_categorical(trainY, 2)
testY = np_utils.to_categorical(testY, 2)
aug = ImageDataGenerator(rotation_range=25, width_shift_range=0.1,
height_shift_range=0.1, shear_range=0.2, zoom_range=0.2,
horizontal_flip=True, fill_mode="nearest")
print("[INFO] load previously trained model")
modelPath = args["loadmodel"]
model = load_model(modelPath)
print("[INFO] training network...")
H = model.fit_generator(
aug.flow(trainX, trainY, batch_size=BS),
validation_data=(testX, testY),
steps_per_epoch=len(trainX) // BS,
epochs=EPOCHS, verbose=1)
print("[INFO] serializing network...")
model.save(args["model"])
# my attempt to keep the labels of all the training session in label binarizer
prevArray = './train_output/previous_data_array.pickle'
arrPickle = labels
if os.path.getsize(prevArray) > 0:
prev = pickle.loads(open(prevArray, 'rb').read())
arrPickle = np.concatenate((prev,labels), axis=0)
lb = LabelBinarizer()
lb.fit_transform(arrPickle)
print("[INFO] serializing combined label array...")
f = open(prevArray, "wb")
f.write(pickle.dumps(arrPickle))
f.close()
print("[INFO] serializing label binarizer...")
f = open(args["labelbin"], "wb")
f.write(pickle.dumps(lb))
f.close()
解决方案
推荐阅读
- javascript - Variable assignment in class scope works in React but not in ES6
- reactjs - onClick 事件时翻转图像
- c - 得到无符号长长加法进位
- r - R:grepl 以错误的顺序输出匹配项
- javascript - 如何使用数组中的特定值显示新窗口
- vue.js - REST API Patch 调用后如何刷新 v-data-table?
- reactjs - 如何解决 TypeError: Cannot read property 'then' of undefined 的问题?
- android - Android Periodic WorkManager 未触发
- r - 为什么 cSplit 返回 TRUE 而不是字符
- ios - var 迟到不醒来