时间戳

首页 > 解决方案 > 有没有办法在 Keras 框架中使用 global_step？

tensorflow - 有没有办法在 Keras 框架中使用 global_step？

问题描述

我正在尝试在框架中重现，下面给出polynomial decay了learning rate decay在Keras框架中实现的Tensorflow框架。

def poly_decay(step, initial_value, decay_period_images_seen):
    """
    Decays a variable using a polynomial law.
    :param step: number of images seen by the network since the beginning of the training.
    :param initial_value: The initial value of the variable to decay..
    :param decay_period_images_seen: the decay period in terms of images seen by the network
    (1 epoch of 10 batches of 6 images each means that 1 epoch = 60 images seen).
    Thus this value must be a multiple of the number of batches
    :return: The decayed variable.
    """

    factor = 1.0 - (tf.cast(step, tf.float32) / float(decay_period_images_seen))
    lrate = initial_value * np.power(factor, 0.9)

    return lrate

Keras 是否为 Keras 提供任何隐藏参数（也许我不知道）global step或者是否有类似的参数global step？或者有没有其他方法可以polynomial learning rate decay在Keras框架中实现？

标签： tensorflowmachine-learningkerasdeep-learning

解决方案

基本上，参数本身作为optimisers.

看看优化器。

sgd = optimizers.SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='mean_squared_error', optimizer=sgd)

所以在这里，你可以只传入poly_decay()作为参数。

通常我们使用time-based decay而不是polynomial decay：

learning_rate = 0.1
decay_rate = learning_rate / epochs
momentum = 0.8
sgd = SGD(lr=learning_rate, momentum=momentum, decay=decay_rate, nesterov=False)

查看此博客以获取更多参考！！

推荐阅读