时间戳

首页 > 解决方案 > 关于数组形状的 Keras 错误，但形状似乎正确

python - 关于数组形状的 Keras 错误，但形状似乎正确

问题描述

我正在尝试使用 Keras 和 python 训练一个简单的模型。文本经过完美预处理。但是当我尝试安装它时，我收到以下错误：

File "main.py", line 47, in <module>
    model.fit(x_train, y_train, batch_size=32, epochs=3)
  File "/home/shamildacoder/.local/lib/python3.6/site-packages/keras/engine/training.py", line 952, in fit
    batch_size=batch_size)
  File "/home/shamildacoder/.local/lib/python3.6/site-packages/keras/engine/training.py", line 789, in _standardize_user_data
    exception_prefix='target')
  File "/home/shamildacoder/.local/lib/python3.6/site-packages/keras/engine/training_utils.py", line 138, in standardize_input_data
    str(data_shape))
ValueError: Error when checking target: expected dense_2 to have shape (121885,) but got array with shape (1000,)

但是print(x_train.shape)，print(y_train.shape)两者都返回(121885, 1000)。我看不出任何理由。

代码：https ://pastebin.com/afnzBf6B

from keras.preprocessing.text import Tokenizer
from keras.layers import Dense
from keras.models import Sequential

data = open('movie_lines.txt', encoding='ISO-8859-1')
lines = [line for line in data]
filtered_lines = []

for line in lines:
    sentence = line.split('+++$+++')[4].strip(' ')
    filtered_lines.append(sentence)

train_size = int(len(filtered_lines) * .8)
train_portion = filtered_lines[:train_size]
test_portion = filtered_lines[train_size:]

x_lines = train_portion[::2]
y_lines = train_portion[1::2]
x_test = test_portion[::2]
y_test = test_portion[1::2]
vocab_size = 1000
print('Prepared data')

def prepare_text(text):
    tokenizer = Tokenizer(num_words=vocab_size)
    tokenizer.fit_on_texts(text)
    matrix = tokenizer.texts_to_matrix(text)
    return matrix


x_train = prepare_text(x_lines)
print('matrixed x')
y_train = prepare_text(y_lines)
print('matrixed y')
print(f'X shape: {x_train.shape}')
print(f'Y shape: {y_train.shape}')

model = Sequential()
model.add(Dense(512, input_shape=(vocab_size,), activation='relu'))
model.add(Dense(len(y_lines), activation='softmax'))

model.compile(
        loss='categorical_crossentropy',
        optimizer='adam',
        metrics=['accuracy',]
        )
print('Created and compiled model')

model.fit(x_train, y_train, epochs=3)

score = model.evaluate(x_test, y_test, batch_size=32, epochs=3)
print('Test Score:'+score[0])
print('Test Accuracy:'+score[1])

标签： pythonmachine-learningneural-networkkerasnlp

解决方案

在预处理阶段，您使用的texts_to_matrix()方法（使用默认参数）将给定的序列作为单热编码格式的矩阵行返回。现在，如果您只想使用 Dense 层从 one-hot 编码序列转到另一个 one-hot 编码序列，则需要将最后一层中的单元数设置为词汇表大小（即矩阵）并sigmoid用作最后一层的激活函数：

model = Sequential()
model.add(Dense(512, input_shape=(vocab_size,), activation='relu'))
model.add(Dense(vocab_size, activation='sigmoid'))

推荐阅读