conv-neural-network - 使用 Keras 和 Bert 词嵌入作为嵌入层的文本分类

Question

拥有 2043 个句子的语料库，可分为 5 类。我使用 Huggingface 的 Automodel 对所有这些句子进行编码，并获得了词嵌入。所以每个句子都表示为 [num_of_tokens(varies), 384(fixed)]。

问题：我为每个句子得到的词嵌入长度各不相同：有些是 70 个标记长，有些是 100 个标记长。那么如何填充这些嵌入呢？Keras 嵌入层中是否有内部方法可以解决这个问题？

eg_embeddings = []
for i in range(len(sentences)):
  encoded_input = tokenizer(sentence[i], padding=True, truncation=True, max_length=150, return_tensors='pt')
  eg_embeddings.append(model(**encoded_input))

eg_embeddings.shape # output:(total sentences = 2043, num_tokens, embedding_dimension = 384)
from keras.layers import Embedding
embedding_layer = Embedding(2043,
                            384,
                            weights=[eg_embeddings],
                            trainable=False)
                            
sequence_input = Input(shape=(,), dtype='int32')
embedded_sequences = embedding_layer(sequence_input)
x = Conv1D()(embedded_sequences)
x = MaxPooling1D()(x)
x = Conv1D()(x)
x = MaxPooling1D()(x)
x = Conv1D()(x)
x = MaxPooling1D()(x)  # global max pooling
x = Flatten()(x)
x = Dense()(x)
preds = Dense(activation='softmax')(x)

model = Model(sequence_input, preds)
model.compile(loss='categorical_crossentropy',
              optimizer='rmsprop',
              metrics=['acc'])

# happy learning!
model.fit(x_train, y_train, validation_data=(x_val, y_val),
          epochs=2, batch_size=128)                            

标签： conv-neural-networktext-classificationembedding