时间戳

首页 > 解决方案 > 将 TensorHub 模型合并到我的 Keras 模型中

python - 将 TensorHub 模型合并到我的 Keras 模型中

问题描述

TensorFlow 2.7、Keras 2.7

我正在尝试将现有的 TFHub 模型用作模型中的层。用自定义 keras 层包裹它，但可能错过了批量大小的一些东西。在下面写了一个简单的版本。下面的模型接收 [ BATCH_SIZE, 224, 224, 3 ]，使用 TFHub 模型生成一个表示，另一个简单层生成另一个表示。然后将两者连接起来，使用密集层并输出 [BATCH_SIZE, 10]。

它适用于批量大小 = 1 但批量大小 > 1 预测和评估工作但 fit 返回与最后一个 Dense 层接收不正确大小输入相关的错误。

简单的模型和调用代码：

def simple_model( ):

    input = Input(shape=(224, 224, 3))

    representation = Conv2D(1,(224,224))(input)

    # Prepare input for depth estimation model
    resized_for_midas = tf.image.resize(input, (384, 384))
    transposed = tf.transpose(resized_for_midas, [0, 3, 1, 2])

    depth_estimation = tfhub.KerasLayer('https://tfhub.dev/intel/midas/v2/2', signature = 'serving_default',
    tags=['serve'])(transposed)
    depth_estimation_reshaped = tf.expand_dims(depth_estimation, axis=-1) # Adding 1 in the end so the shape will be [batch_size, 384, 384, 1]
    depth_estimation_repmat = Conv2D(3,(3,3),padding='same')(depth_estimation_reshaped) # Repeat the depth estimation single channel 3 times to match ResNet input using a convolution layer
    depth_estimation_resized = tf.image.resize(depth_estimation_repmat, (224, 224))

    depth_estimation_representation = Conv2D(1,(224,224))(depth_estimation_resized)

    # Concatenate representations
    representation_full = tf.concat([representation, depth_estimation_representation], axis=-1)
    
    flat = tf.reshape(representation_full, (-1, representation_full.shape[-1]))

    # Outputs
    output = Dense(10, input_shape=(-1, flat.shape[-1]),
        activation='linear')(flat)

    model = Model(inputs=input, outputs=output)
    
    return model

model = simple_model()
model.compile(loss='mae')
batch_size = 1
model(np.random.rand(batch_size, 224, 224, 3)) # Works
model.evaluate(np.random.rand(batch_size, 224, 224, 3), np.random.rand(batch_size, 10)) # Works
model.fit(np.random.rand(batch_size, 224, 224, 3), np.random.rand(batch_size, 10)) # Works
batch_size = 2
model(np.random.rand(batch_size, 224, 224, 3)) # Works
model.evaluate(np.random.rand(batch_size, 224, 224, 3), np.random.rand(batch_size, 10)) # Works
model.fit(np.random.rand(batch_size, 224, 224, 3), np.random.rand(batch_size, 10)) # Fails

错误：

回溯（最后一次调用）：文件“”，第 1 行，在文件“/home/dani/projects/venv/lib/python3.8/site-packages/wandb/integration/keras/keras.py”中，第 150 行，在 new_v2 返回 old_v2(*args, **kwargs) 文件“/home/dani/projects/venv/lib/python3.8/site-packages/keras/utils/traceback_utils.py”，第 67 行，在 error_handler raise e .with_traceback(filtered_tb) 来自无文件“/home/dani/projects/venv/lib/python3.8/site-packages/tensorflow/python/eager/execute.py”，第 58 行，在 quick_execute tensors = pywrap_tfe.TFE_Py_Execute( ctx._handle, device_name, op_name, tensorflow.python.framework.errors_impl.InvalidArgumentError: 矩阵大小不兼容: In[0]: [2,2], In[1]: [1,10] [[node gradient_tape/model_1 /dense_1/MatMul/MatMul_1（定义在 /home/dani/projects/venv/lib/python3.8/site-packages/keras/optimizer_v2/optimizer_v2.py:464) ]] [Op:__inference_train_function_30163]

错误可能源于输入操作。连接到节点 gradient_tape/model_1/dense_1/MatMul/MatMul_1 的输入源操作：In[0] model_1/tf.reshape_1/Reshape（定义在 /home/dani/projects/venv/lib/python3.8/site-packages/keras /layers/core/tf_op_layer.py:261)
In[1] gradient_tape/mean_absolute_error/sub/Reshape：

标签： pythontensorflowkeras

解决方案

@Dani 这是因为tfhub.KerasLayer您在代码中使用的是为单个图像构建的。

在那个 tfhub 页面https://tfhub.dev/intel/midas/v2/2中明确提到了

用于从单个 RGB 图像进行单目深度估计的卷积神经网络。

另外，检查model.summary哪个显示差异

Model: "model"
__________________________________________________________________________________________________
 Layer (type)                   Output Shape         Param #     Connected to                     
==================================================================================================
 input_1 (InputLayer)           [(None, 224, 224, 3  0           []                               
                                )]                                                                
                                                                                                  
 tf.image.resize (TFOpLambda)   (None, 384, 384, 3)  0           ['input_1[0][0]']                
                                                                                                  
 tf.compat.v1.transpose (TFOpLa  (None, 3, 384, 384)  0          ['tf.image.resize[0][0]']        
 mbda)                                                                                            
                                                                                                  
 keras_layer (KerasLayer)       (1, 384, 384)        0           ['tf.compat.v1.transpose[0][0]'] 
                                                                                                  
 tf.expand_dims (TFOpLambda)    (1, 384, 384, 1)     0           ['keras_layer[0][0]']            
                                                                                                  
 conv2d_1 (Conv2D)              (1, 384, 384, 3)     30          ['tf.expand_dims[0][0]']         
                                                                                                  
 tf.image.resize_1 (TFOpLambda)  (1, 224, 224, 3)    0           ['conv2d_1[0][0]']               
                                                                                                  
 conv2d (Conv2D)                (None, 1, 1, 1)      150529      ['input_1[0][0]']                
                                                                                                  
 conv2d_2 (Conv2D)              (1, 1, 1, 1)         150529      ['tf.image.resize_1[0][0]']      
                                                                                                  
 tf.concat (TFOpLambda)         (1, 1, 1, 2)         0           ['conv2d[0][0]',                 
                                                                  'conv2d_2[0][0]']               
                                                                                                  
 tf.reshape (TFOpLambda)        (1, 2)               0           ['tf.concat[0][0]']              
                                                                                                  
 dense (Dense)                  (1, 10)              30          ['tf.reshape[0][0]']             
                                                                                                  
==================================================================================================
Total params: 301,118
Trainable params: 301,118
Non-trainable params: 0
__________________________________________________________________________________________________

请在此处查看要点

我想你需要找到一个支持多个图像作为批处理的层。谢谢！

推荐阅读