时间戳

首页 > 解决方案 > 在 keras 自定义层中包含高级计算（类 scikit）

python - 在 keras 自定义层中包含高级计算（类 scikit）

问题描述

通常我会在将数据输入模型进行分类之前对其进行预处理。

然而，这是不可能的，因此要么进一步（以某种方式）增强模型的性能，要么直接在模型内部包含有用的预处理步骤。

我怎样才能做到这一点？到目前为止，我发现的最佳解决方案包括使用 Keras 后端重新实现我想要的功能。这远非一个好的解决方案，因此我希望有人有一个想法，如何挽救这种情况。

以下是我发现有用的链接 + 我当前的代码。

有用的链接：

具有高级计算功能的 Keras 自定义层

如何从 Keras Tensortype 切换到自定义层的 numpy 数组？

如何使用后端未包含的功能创建 Keras 自定义层以执行张量采样？

到目前为止我的代码：

def freezeBaseModelLayers(baseModel):
    for layer in baseModel.layers:
        layer.trainable = False


def preprocess_input(x):
    # TODO: Not working, but intention should be clear
    numpy_array = tf.unstack(tf.unstack(tf.unstack(x, 224, 0), 224, 0), 1, 0)
    from skimage.feature import hog
    from skimage import data, exposure
    img_adapteq = exposure.equalize_adapthist(numpy_array, orientations=8, pixels_per_cell=(3, 3),
                                              cells_per_block=(1, 1), visualize=True, multichannel=False)
    [x1, x2, x3] = tf.constant(img_adapteq), tf.constant(img_adapteq), tf.constant(img_adapteq)
    img_conc = Concatenate([x1, x2, x3])
    return img_conc


def create(x):
    is_training = tf.get_variable('is_training', (), dtype=tf.bool, trainable=False)
    with tf.name_scope('pretrained'):
        # Add preprocess step here...
        input_layer = Lambda(preprocess_input(x), input_shape=(224, 224, 1), output_shape=(224, 224, 3))

        baseModel = vgg16.VGG16(weights='imagenet', include_top=False, input_shape=(224, 224, 3))    
        freezeBaseModelLayers(baseModel)
        layer = baseModel(input_layer)
        layer = GlobalMaxPooling2D()(layer)
        layer = Dense(1024, activation='relu')(layer)
        layer = Dense(2, activation=None)(layer)
        model = Model(input=input_layer.input, output=layer)
        output = model(x)
        return output




I would like to include prepocessing steps inside my model

The models I am working with are receiving noisy data. In order to enhance the performance of the models, I would like to do some preprocessing steps e.g. equalize_adapthist.

标签： pythonnumpytensorflowkerasscikit-image

解决方案

更好的方法是通过自定义 keras 层。这是一个例子：

import tensorflow as tf
from keras.layers import Layer, Input, Conv2D
from keras.models import Model
from keras import backend as K
from skimage.feature import hog
from skimage import data, exposure

def equalize(img):
  img_adapteq = exposure.equalize_adapthist(img)
  return img_adapteq

def preprocess_input(img):
  return tf.py_func(equalize, 
                     [img],
                     'float32',
                     stateful=False,
                     name='custom_image_op')


class CustomLayer(Layer):
  def __init__(self, output_dim, **kwargs):
    self.output_dim = output_dim
    self.trainable = False
    super(CustomLayer, self).__init__(**kwargs)

  def call(self, x):
    res = tf.map_fn(preprocess_input, x)
    res.set_shape([x.shape[0],
                   self.output_dim[1], 
                   self.output_dim[0],
                   x.shape[-1]])
    return res

output_dim = (224,224)
inputs = Input(shape=(224,224,3))
x = CustomLayer(output_dim)(inputs)
x = Conv2D(32, (3,3))(x)
x = Flatten()(x)
x = Dense(1)(x)

model = Model(inputs, x)
model.summary()

# test
sample = np.random.rand(4, 224,224,3).astype(np.float32)
y = np.random.randint(2, size=(4,))

model.compile("sgd", "mse")
model.fit(sample, y)

推荐阅读