python - Keras Python 脚本有时运行良好,有时因矩阵大小不兼容而失败:In[0]: [10000,1], In[1]: [3,1]
问题描述
我希望 Keras 能够识别自拍和非自拍,首先我只使用 4 张图片进行开发,然后再使用完整数据。
问题:脚本有时会正常运行并退出,有时会失败并出现Matrix size-incompatible: In[0]: [10000,1], In[1]: [3,1]
以下错误:
$ python run.py
TensorFlow version: 1.10.1
file_names: ['selfies-dev-data/0/lego.jpg', 'selfies-dev-data/0/dakota.jpg', 'selfies-dev-data/1/ammar.jpg', 'selfies-dev-data/1/olivier.jpg']
labels: [0.0, 0.0, 1.0, 1.0]
dataset: <PrefetchDataset shapes: ((?, 100, 100, 1), (?, 1)), types: (tf.float32, tf.float32)>
images: Tensor("IteratorGetNext:0", shape=(?, 100, 100, 1), dtype=float32)
labels: Tensor("IteratorGetNext:1", shape=(?, 1), dtype=float32)
Epoch 1/1
2018-09-25 13:59:17.285143: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
1/2 [==============>...............] - ETA: 0s - loss: 0.7741 - acc: 0.0000e+00Traceback (most recent call last):
File "run.py", line 64, in <module>
model.fit(images, labels, epochs=1, steps_per_epoch=2)
File "/home/nico/.local/lib/python2.7/site-packages/tensorflow/python/keras/engine/training.py", line 1363, in fit
validation_steps=validation_steps)
File "/home/nico/.local/lib/python2.7/site-packages/tensorflow/python/keras/engine/training_arrays.py", line 205, in fit_loop
outs = f(ins)
File "/home/nico/.local/lib/python2.7/site-packages/tensorflow/python/keras/backend.py", line 2914, in __call__
fetched = self._callable_fn(*array_vals)
File "/home/nico/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1382, in __call__
run_metadata_ptr)
File "/home/nico/.local/lib/python2.7/site-packages/tensorflow/python/framework/errors_impl.py", line 519, in __exit__
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Matrix size-incompatible: In[0]: [10000,1], In[1]: [3,1]
[[Node: rgb_to_grayscale/Tensordot/MatMul = MatMul[T=DT_FLOAT, transpose_a=false, transpose_b=false](rgb_to_grayscale/Tensordot/Reshape, rgb_to_grayscale/Tensordot/Reshape_1)]]
[[Node: IteratorGetNext = IteratorGetNext[output_shapes=[[?,100,100,1], [?,1]], output_types=[DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](OneShotIterator)]]
如何调查问题?代码将图像大小调整为100*100,输入层也是100*100,不知道从哪里来3
。
作为参考,这是我的源代码:
import os
import tensorflow as tf
print "TensorFlow version: " + tf.__version__
out_shape = tf.convert_to_tensor([100, 100])
batch_size = 2
data_folders = ["selfies-dev-data/0", "selfies-dev-data/1"]
classes = [0., 1.]
epoch_size = len(data_folders)
file_names = [] # Path of all data files
labels = [] # Label of each data file (same size as the array above)
for d, l in zip(data_folders, classes):
name = [os.path.join(d,f) for f in os.listdir(d)] # get the list of all the images file names
file_names.extend(name)
labels.extend([l] * len(name))
print "file_names: " + str(file_names)
print "labels: " +str(labels)
file_names = tf.convert_to_tensor(file_names, dtype=tf.string)
labels = tf.convert_to_tensor(labels)
dataset = tf.data.Dataset.from_tensor_slices((file_names, labels))
dataset = dataset.repeat().shuffle(epoch_size)
def map_fn(path, label):
# path/label represent values for a single example
image = tf.image.decode_jpeg(tf.read_file(path))
# some mapping to constant size - be careful with distorting aspect ratios
image = tf.image.resize_images(image, out_shape)
image = tf.image.rgb_to_grayscale(image)
# color normalization - just an example
image = tf.to_float(image) * (2. / 255) - 1
label = tf.expand_dims(label, axis=-1)
return image, label
# num_parallel_calls > 1 induces intra-batch shuffling
dataset = dataset.map(map_fn, num_parallel_calls=8)
dataset = dataset.batch(batch_size)
dataset = dataset.prefetch(1)
print "dataset: " + str(dataset)
images, labels = dataset.make_one_shot_iterator().get_next()
# Following is from https://www.tensorflow.org/tutorials/keras/basic_classification
from tensorflow import keras
model = keras.Sequential([
keras.layers.Flatten(input_shape=(100, 100, 1)),
keras.layers.Dense(128, activation=tf.nn.relu),
keras.layers.Dense(1, activation=tf.nn.sigmoid)
])
model.compile(optimizer=tf.train.AdamOptimizer(),
loss='binary_crossentropy',
metrics=['accuracy'])
print "images: " + str(images)
print "labels: " + str(labels)
model.fit(images, labels, epochs=1, steps_per_epoch=2)
解决方案
问题似乎在以下行:
image = tf.image.rgb_to_grayscale(image)
tf.image.rgb_to_grayscale
期望给定的图像张量具有大小为 3 的最后一维,表示 RGB 通道。但是,该行:
image = tf.image.decode_jpeg(tf.read_file(path))
可以产生具有不同数量通道的张量。这是因为tf.image.decode_jpeg
默认情况下,将创建一个通道数与 JPEG 数据中的通道数相同的张量。因此,如果您的图像已经是灰度图像,那么张量将只有一个通道,并且程序将失败。您可以通过在所有情况下请求将 JPEG 数据解码为 RGB 图像来解决该问题,将channels
参数设置为3
:
image = tf.image.decode_jpeg(tf.read_file(path), channels=3)
这将确保您的所有图像都得到统一处理。
推荐阅读
- python - 通过 GoogleOAuth2 使用 is_active=False 登录的 Django 用户
- python - 当 api 需要使用 Python 的 requests 包进行两因素身份验证时,如何发出 put 请求?
- c# - 在实体框架中使用 UserName 作为外键创建 ApplicationUser 导航属性
- mysql - Mysql "like"-query 没有结果,但它应该
- css - CSS 类在内部 CSS 类之前应用并且不影响 SemanticUI 按钮
- mysql - 如何在sequelize模型中获得继承
- python - 在另一个函数中使用 Tensorflow 会话时找不到变量
- javascript - 如何在不使用服务的情况下将子组件之间的反应式表单数据传递给父组件
- kotlin - Kotlin 的函数式主函数参数解析
- powershell - 如何处理复制项的错误