时间戳

首页 > 解决方案 > Tensorflow 2.1 来自生成器的数据集 - ValueError：检查输入时出错：预期 Geo 的形状为（504，）但得到的数组形状为（1，）

python - Tensorflow 2.1 来自生成器的数据集 - ValueError：检查输入时出错：预期 Geo 的形状为（504，）但得到的数组形状为（1，）

问题描述

我无法编译我的模型，我似乎经常遇到这个问题，并且错误输出并不是特别有用：

预计 Geo 的形状为 (504,) 但得到的数组形状为 (1,)

我知道形状在某处不匹配，但此错误并未将我指向代码中的任何特定点。是来自 Input() 层、第一个 Dense 层还是来自 Generator 本身？

我的模型是一个多输入单输出模型，它在“Geo”输入中采用 504 个浮点输入的 numpy 数组，在“SDF”输入中采用 64、64、64、1 的 SDF。这些被连接起来输出一个 12 浮点数的结果。

我的逻辑如下：

# The Data Generator
# Input shape is 42, 12, 1
# SDF shape is 64, 64, 64

def data_generator(train_path, test_path):
    """
    Generator for loading geo data.
    """

    imgListTestSDF = test_path.parent
    imgListTestSDF = imgListTestSDF / "test_sdf"    

    imgListTrain = sorted(train_path.iterdir() )
    imgListTest = sorted(test_path.iterdir() )
    imgListTestSDF = sorted(imgListTestSDF.iterdir() )

    for dist, norm, norm_sdf in zip(imgListTrain, imgListTest, imgListTestSDF):
        norm = np.load(norm)['array']
        norm_sdf = np.load(norm_sdf)['array']
        dist = np.load(dist)['array']

        outshape = norm.shape[0] * 3 * 12
        print ("Out shape: %s" % (outshape,)) # Out shape: 504
        #norm = np.reshape(norm, (504))
        norm = norm.flatten()
        print ("Norm reshaped: %s" % (norm.shape,)) # Norm reshaped: (504,)

        norm_sdf = np.reshape(norm_sdf, IN_SDF_SHAPE)

        # Cast to float32, loaded as float64
        norm = norm.astype(np.float32)
        dist = dist.astype(np.float32)

        norm_combined = {}
        norm_combined['Geo'] = norm
        norm_combined['SDF'] = norm_sdf

        yield norm_combined, dist

神经网络逻辑：

EPOCHS = 100
BATCH_SIZE = 10

IN_GEO_SHAPE = [504]
inGeoTensorShape = tf.TensorShape(IN_GEO_SHAPE)

IN_SDF_SHAPE = [64, 64, 64, 1]
inSDFTensorShape = tf.TensorShape(IN_SDF_SHAPE)

outTypes = ({'Geo': tf.dtypes.float32 , 'SDF' : tf.dtypes.float32}, tf.dtypes.float32)
outTensorShape = ({'Geo': inGeoTensorShape , 'SDF' : inSDFTensorShape }, tf.TensorShape([12,]))

trainGen = partial(data_generator, train, test) 
dataset = tf.data.Dataset.from_generator(trainGen, outTypes, output_shapes=outTensorShape )


# Geo Dense layers.
inGeo = tf.keras.Input(shape=inGeoTensorShape, batch_size=BATCH_SIZE, name="Geo" )
print ("In Geo layer shape: %s" % (inGeo.shape,)) # In Geo layer shape: (10, 504)

dense1 = tf.keras.layers.Dense(512)(inGeo)
dense2 = tf.keras.layers.Dense(512)(dense1)
dense3 = tf.keras.layers.Dense(512)(dense2)
geoOut = tf.keras.layers.Flatten()(dense3)

print ("Geo Out Shape: %s" % (geoOut.shape, )) # Geo Out Shape: (10, 512)

# SDF CNN Net
initializer = tf.random_normal_initializer(0.0, 1.0)

inSDF = tf.keras.Input(shape=inSDFTensorShape, batch_size=BATCH_SIZE, name="SDF")
print ("In SDF Shape: %s" % (inSDF.shape,) )  # In SDF Shape: (10, 64, 64, 64, 1)
conv1 = tf.keras.layers.Conv3D(32, 4, padding='same', kernel_initializer=initializer, input_shape=inSDFTensorShape)(inSDF) # 32
max1 = tf.keras.layers.MaxPool3D((2, 2, 2))(conv1)
... Many CNN layers ...
conv6 = tf.keras.layers.Conv3D(512, 4, padding='same', kernel_initializer=initializer)(max5) # 1
max6 = tf.keras.layers.MaxPool3D((2, 2, 2))(conv6)
sdfOut = tf.keras.layers.Flatten()(max6)

print ("SDF Out Shape: %s" % (sdfOut.shape, )) # SDF Out Shape: (10, 512)

# Concatenation and Output

concat = tf.keras.layers.concatenate([geoOut, sdfOut])
decode1 = tf.keras.layers.Dense(512)(concat)
decode2 = tf.keras.layers.Dense(128)(decode1)
output = tf.keras.layers.Dense(12)(decode2)

print ("Creating Model")
model = tf.keras.Model(inputs=[inGeo, inSDF], outputs=output ) 
model.summary()

optimiser = tf.keras.optimizers.Adam()
#loss = tf.keras.losses.MSE()

print("Compiling Model")
model.compile(optimizer=optimiser, loss='mse', metrics=['accuracy'])

print("Training Model")
#model.fit(data_gener#ator(train, test) , epochs=EPOCHS, steps_per_epoch=30 )
model.fit(dataset, epochs=EPOCHS, steps_per_epoch=10 )

这是模型摘要：

Model: "model"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to
==================================================================================================
SDF (InputLayer)                [(10, 64, 64, 64, 1) 0
__________________________________________________________________________________________________
conv3d (Conv3D)                 (10, 64, 64, 64, 32) 2080        SDF[0][0]
__________________________________________________________________________________________________
max_pooling3d (MaxPooling3D)    (10, 32, 32, 32, 32) 0           conv3d[0][0]
__________________________________________________________________________________________________
conv3d_1 (Conv3D)               (10, 32, 32, 32, 128 262272      max_pooling3d[0][0]
__________________________________________________________________________________________________
max_pooling3d_1 (MaxPooling3D)  (10, 16, 16, 16, 128 0           conv3d_1[0][0]
__________________________________________________________________________________________________
conv3d_2 (Conv3D)               (10, 16, 16, 16, 256 2097408     max_pooling3d_1[0][0]
__________________________________________________________________________________________________
max_pooling3d_2 (MaxPooling3D)  (10, 8, 8, 8, 256)   0           conv3d_2[0][0]
__________________________________________________________________________________________________
conv3d_3 (Conv3D)               (10, 8, 8, 8, 512)   8389120     max_pooling3d_2[0][0]
__________________________________________________________________________________________________
max_pooling3d_3 (MaxPooling3D)  (10, 4, 4, 4, 512)   0           conv3d_3[0][0]
__________________________________________________________________________________________________
Geo (InputLayer)                [(10, 504)]          0
__________________________________________________________________________________________________
conv3d_4 (Conv3D)               (10, 4, 4, 4, 512)   16777728    max_pooling3d_3[0][0]
__________________________________________________________________________________________________
dense (Dense)                   (10, 512)            258560      Geo[0][0]
__________________________________________________________________________________________________
max_pooling3d_4 (MaxPooling3D)  (10, 2, 2, 2, 512)   0           conv3d_4[0][0]
__________________________________________________________________________________________________
dense_1 (Dense)                 (10, 512)            262656      dense[0][0]
__________________________________________________________________________________________________
conv3d_5 (Conv3D)               (10, 2, 2, 2, 512)   16777728    max_pooling3d_4[0][0]
__________________________________________________________________________________________________
dense_2 (Dense)                 (10, 512)            262656      dense_1[0][0]
__________________________________________________________________________________________________
max_pooling3d_5 (MaxPooling3D)  (10, 1, 1, 1, 512)   0           conv3d_5[0][0]
__________________________________________________________________________________________________
flatten (Flatten)               (10, 512)            0           dense_2[0][0]
__________________________________________________________________________________________________
flatten_1 (Flatten)             (10, 512)            0           max_pooling3d_5[0][0]
__________________________________________________________________________________________________
concatenate (Concatenate)       (10, 1024)           0           flatten[0][0]
                                                                 flatten_1[0][0]
__________________________________________________________________________________________________
dense_3 (Dense)                 (10, 512)            524800      concatenate[0][0]
__________________________________________________________________________________________________
dense_4 (Dense)                 (10, 128)            65664       dense_3[0][0]
__________________________________________________________________________________________________
dense_5 (Dense)                 (10, 12)             1548        dense_4[0][0]
==================================================================================================
Total params: 45,682,220
Trainable params: 45,682,220
Non-trainable params: 0
__________________________________________________________________________________________________

这是完整的错误（不包括完整的堆栈跟踪）：

ValueError: in converted code:

    C:\Python37\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py:677 map_fn
        batch_size=None)
    C:\Python37\lib\site-packages\tensorflow_core\python\keras\engine\training.py:2410 _standardize_tensors
        exception_prefix='input')
    C:\Python37\lib\site-packages\tensorflow_core\python\keras\engine\training_utils.py:582 standardize_input_data
        str(data_shape))

    ValueError: Error when checking input: expected Geo to have shape (504,) but got array with shape (1,)

最后，如果有人能指出我哪里出错了，那将不胜感激。如果他们也能帮助我理解为什么我会收到这个错误，我将永远感激不尽。到目前为止，理解 NN 和 TF 中的形状让我感到困惑。

谢谢！

标签： pythontensorflowmachine-learningtensorflow2.0tensorflow-datasets

解决方案

Geo layer期望输入大小，(504,)但您给它输入大小(1,)

也许当您在其中进行操作norm = norm.flatten()时，data_generator它正在转换为一个大小为 1 的数组，其中包含 504 个元素。所以也许尝试将其重塑为(504,)

推荐阅读