时间戳

首页 > 解决方案 > 如何在 .csv 文件的数据集上训练我的 CNN 模型？

python - 如何在 .csv 文件的数据集上训练我的 CNN 模型？

问题描述

大家好，我是 Python 初学者，我正在使用 google Colab 训练我的第一个 CNN 模型，我在训练部分受阻，我知道我必须使用 model.fit() 来训练模型，但我有不知道 model.fit() 里面有什么，另外我不确定从 .csv 文件中拆分我的数据集的部分。这是我的代码：

from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D
from keras.utils import to_categorical
from keras.preprocessing import image
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from keras.utils import to_categorical
from sklearn.metrics import accuracy_score
from tqdm import tqdm
from numpy.random import RandomState
import csv

df = pd.read_csv('classes.csv')
print(df.head(3500))
#splitting the dataset in training and testing sets.
rng = RandomState()

train = df.sample(frac=0.7, random_state=rng)
test = df.loc[~df.index.isin(train.index)]



#model's structure

model = Sequential()
#convolutional layer
model.add(Conv2D(32, kernel_size=(3, 3),activation='relu',input_shape=(28,28,1)))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
#flatten output of conv
model.add(Flatten())
#hidden layer
model.add(Dense(128, activation='relu')) 
#output layer
model.add(Dense(10, activation='softmax')) 
#compiling sequential model
model.compile(loss='categorical_crossentropy',optimizer='Adam',metrics=['accuracy'])
model.summary()
#training the model
model.fit()

标签： pythoncsvtensorflowconv-neural-networktraining-data

解决方案

好的，如果我们能提供帮助，让我们开始吧。首先是关于拆分数据框。最好将其拆分为训练集、验证集和测试集。最好的方法是使用 sklearn 的 train, test, split。文档是 [here.][1] 使用下面的代码将数据 Trame 分成 3 组

from sklearn.model_selection import train_test_split
train_split=.8  # percentage of df to use for training
test_split = .05 # percentage of df to use for test
# note valid_split=1- train_split = test_ split in this case .15
train_df, dummy_df= train_test_split(df, train_split=train_split, shuffle=True, random_state=123)
dummy_split=test_split/(1.0 - train_split)
test_df, valid_df=train_test_split(dummy_df, train_split=dummy_split) # note dummy_df is 20% of df

现在您需要创建训练、测试和验证生成器。我假设您有图像输入。使用 ImageDataGenerator.flow_from_dataframe() 创建您的生成器。相关文档是 [here.][2] 通常在 0 到 1 或 -1 到 +1 范围内缩放像素图像。我更喜欢后者。下面是生成器的代码

def scalar(img):
       return img/127.5 -1
x_col='Filename' # set this to point to the df column that holds the full path to the image file
y_col- 'Label' # set this to the df column that holds the class labels
batch_size = 32 # set this to the batch size
class_mode='categorical'
image_shape=(64,64) # set this to desired image size
tgen=tf.keras.preprocessing.image.ImageDataGenerator(preprocessing_function=scalar, horizontal_flip=True)
gen= tf.keras.preprocessing.image.ImageDataGenerator(preprocessing_function=scalar)
train_gen=tgen.flow_from_dataframe(train_df, x_col=x_col, 
                                   y_col=y_col,target_size=image_shape,
                                   class_mode=class_mode, batch_size=batch_size, 
                                    shuffle=True, random_state=123)                                   
test_gen=tgen.flow_from_dataframe(test_df, x_col= x_col, y_col=y_col 
                                   class_mode=class_mode, 
                                   batch_size=batch_size, shuffle=False)                                        
valid_gen=tgen.flow_from_dataframe(valid_df, x_col=x_col, shuffle=False,
                                   y_col=y_col,batch_size=batch_size 
                                   class_mode=class_mode , target_size=image_shape)

现在是时候使用 model.fit 训练你的模型了。文档在[这里][3]

epochs = 15 # set this to the number of epochs to run
history=model.fit(train_gen, epochs=epochs, validation_data= valid_gen, verbose=1)

当模型完成训练后，你想看看它在测试集上的表现如何。你这样做是为了 model.evaluate 如下所示

accuracy = model.evaluate(test_gen, verbose=1)[1]
print (accuracy)

您可以使用您的模型使用 model.predict 进行预测

preds=model.predict(test_gen, verbose=1)

preds 是一个矩阵，其行数等于测试集中的样本数，列数等于您拥有的类数。每个条目都是图像属于该类的概率值。您要选择概率值最高的列。使用 np.argmax 查找具有最高概率值的列（类）。您可以使用 labels-test_gen.labels 获取 test_set 标签。代码如下

correct_preds=0
for i in range (len(preds):
    index=np.argmax(preds[i]) # column with highest probability
    label=test_gen.labels[i] # true class of image
    if index == label:
        correct_preds +=1
pred_accuracy= correct_preds/ len(preds)
print ( pred_accuracy)
    
                                
 


  [1]: https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html
  [2]: https://keras.io/api/preprocessing/image/
  [3]: https://www.tensorflow.org/api_docs/python/tf/keras/Model

推荐阅读