python-3.x - 在 tensorflow2 中使用自定义损失和自定义训练

Question

我想预测一条曲线 y = x * w = b。但是，我希望你使用 loss = tf.abs(dy_true - dy_pred) + tf.abs(y_true - y_pred)。我用的是tensorflow2.3，怎么办。

import matplotlib.pyplot as plt
import tensorflow as tf
import numpy as np
import time
from tensorflow.keras import *

inputs = layers.Input(shape=(3,))
x = layers.Dense(64, activation="elu")(inputs)
x = layers.Dense(128, activation="elu")(x)
outputs = layers.Dense(1)(x)
model = models.Model(inputs=inputs, outputs=outputs)
model_1 = models.Model(inputs=inputs, outputs=outputs)
opt = optimizers.Adam(0.00001)

model_1.compile(optimizer=opt, loss="mse")


# randomly generate data
x = np.random.normal(0, 1, (100, 3))
y = np.dot(x, np.array([1,2,1])).reshape(100, 1)
y = y + np.random.normal(0, 0.1, (100, 1))

# model_1
start_t = time.time()
model_1.fit(x=x,y=y, epochs=50)
end_t1 = time.time() - start_t


# model
# I want yo use: loss = tf.abs(dy_true - dy_pred) + tf.abs(y_true - y_pred).
start_t = time.time()
for i in range(50):
    if i < 40:
        alpha = 0
    else:
        alpha = 0.8
    with tf.GradientTape() as tape:
        pred = model(x)
        a = pred.numpy()
        d_pred = np.gradient(a, axis=0)
        d_true = np.gradient(y, axis=0)
        loss_d = tf.cast(tf.abs(d_true - d_pred), tf.float32)
        loss = tf.abs(y-pred) * (1-alpha) + loss_d * alpha
    gradients = tape.gradient(loss, model.trainable_variables)
    opt.apply_gradients(zip(gradients, model.trainable_variables))
end_t2 = time.time() - start_t

print("time of model:", end_t2,
      "\ntime of model_1:", end_t1)

plt.clf()
plt.plot(y[90:], "g")
plt.plot(model(x[90:]), "r")
plt.plot(model_1.predict(x[90:]) - 0.1, "y")
plt.show()

现在，我有两个问题。首先，我想用Model.fit的内置方法来实现Loss，但是一直没有找到方法。其次，自定义训练的时间比model.fit要长很多。当我将自定义训练的方法应用于大量数据时，出现错误：tensorflow.python.Framework.errors_impl。ResourceExhaustedError：分配形状为 [205, 1000,256] 的张量并通过分配器 GPU_0_bfc [Op:EluGrad] 在 /job:localhost/replica:0/task:0/device:GPU:0 上键入 float 时出现 OOM。

标签： python-3.xtensorflow2.0