python - keras 验证精度和人工评估精度的巨大差异。(Python3 Keras)
问题描述
keras model.evaluate 和我的手动评估中似乎存在巨大的精度差异。我无法解释。
这些是训练和 keras 评估日志。
Training:
* dataset: /etc/crypto/dataset/exchange_alpaca_symbols_250-seq_length_4-width_128-height_128-scale_2.5-ratio_0.33_0.33_0.33/
* train samples: 37797
* validation samples: 9450
Found 37797 images belonging to 3 classes.
* class_indices: {'bearish_1': 0, 'bullish_1': 1, 'normal': 2}
Found 9450 images belonging to 3 classes.
2021-08-11 22:14:19.374432: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:176] None of the MLIR Optimization Passes are enabled (registered 2)
2021-08-11 22:14:19.393402: I tensorflow/core/platform/profile_utils/cpu_utils.cc:114] CPU Frequency: 3999980000 Hz
Epoch 1/50
590/590 [==============================] - 1040s 2s/step - loss: 0.9058 - accuracy: 0.5142 - precision: 0.6863 - recall: 0.2499 - val_loss: 0.8197 - val_accuracy: 0.5663 - val_precision: 0.8139 - val_recall: 0.2454
Epoch 2/50
590/590 [==============================] - 1036s 2s/step - loss: 0.8245 - accuracy: 0.5619 - precision: 0.6964 - recall: 0.3257 - val_loss: 0.8118 - val_accuracy: 0.5654 - val_precision: 0.6212 - val_recall: 0.4641
Epoch 3/50
590/590 [==============================] - 1036s 2s/step - loss: 0.8114 - accuracy: 0.5746 - precision: 0.6957 - recall: 0.3554 - val_loss: 0.7968 - val_accuracy: 0.5804 - val_precision: 0.7403 - val_recall: 0.3260
Epoch 4/50
590/590 [==============================] - 1035s 2s/step - loss: 0.8010 - accuracy: 0.5798 - precision: 0.6948 - recall: 0.3704 - val_loss: 0.7930 - val_accuracy: 0.5828 - val_precision: 0.6785 - val_recall: 0.4072
Epoch 5/50
590/590 [==============================] - 1035s 2s/step - loss: 0.7922 - accuracy: 0.5858 - precision: 0.6960 - recall: 0.3880 - val_loss: 0.7958 - val_accuracy: 0.5824 - val_precision: 0.6376 - val_recall: 0.4689
Epoch 6/50
590/590 [==============================] - 1035s 2s/step - loss: 0.7851 - accuracy: 0.5961 - precision: 0.6919 - recall: 0.4051 - val_loss: 0.7850 - val_accuracy: 0.5914 - val_precision: 0.7227 - val_recall: 0.3543
Epoch 7/50
590/590 [==============================] - 1036s 2s/step - loss: 0.7786 - accuracy: 0.6022 - precision: 0.6922 - recall: 0.4277 - val_loss: 0.7821 - val_accuracy: 0.5867 - val_precision: 0.6624 - val_recall: 0.4311
Epoch 8/50
590/590 [==============================] - 1035s 2s/step - loss: 0.7696 - accuracy: 0.6108 - precision: 0.6939 - recall: 0.4522 - val_loss: 0.7833 - val_accuracy: 0.5968 - val_precision: 0.6905 - val_recall: 0.4226
Epoch 9/50
590/590 [==============================] - 1035s 2s/step - loss: 0.7602 - accuracy: 0.6156 - precision: 0.6950 - recall: 0.4658 - val_loss: 0.7878 - val_accuracy: 0.5897 - val_precision: 0.6951 - val_recall: 0.3885
Epoch 10/50
590/590 [==============================] - 1035s 2s/step - loss: 0.7538 - accuracy: 0.6270 - precision: 0.6962 - recall: 0.4902 - val_loss: 0.7896 - val_accuracy: 0.5997 - val_precision: 0.6665 - val_recall: 0.4549
Epoch 11/50
590/590 [==============================] - 1035s 2s/step - loss: 0.7440 - accuracy: 0.6294 - precision: 0.6948 - recall: 0.5093 - val_loss: 0.7810 - val_accuracy: 0.6007 - val_precision: 0.6604 - val_recall: 0.4655
Epoch 12/50
590/590 [==============================] - 1035s 2s/step - loss: 0.7373 - accuracy: 0.6372 - precision: 0.7013 - recall: 0.5250 - val_loss: 0.7921 - val_accuracy: 0.5931 - val_precision: 0.6419 - val_recall: 0.4968
Epoch 13/50
590/590 [==============================] - 1035s 2s/step - loss: 0.7268 - accuracy: 0.6464 - precision: 0.7018 - recall: 0.5436 - val_loss: 0.8048 - val_accuracy: 0.5979 - val_precision: 0.6531 - val_recall: 0.4844
Epoch 14/50
590/590 [==============================] - 1035s 2s/step - loss: 0.7169 - accuracy: 0.6541 - precision: 0.7044 - recall: 0.5647 - val_loss: 0.8292 - val_accuracy: 0.5998 - val_precision: 0.6489 - val_recall: 0.5046
Epoch 15/50
590/590 [==============================] - 1035s 2s/step - loss: 0.7064 - accuracy: 0.6629 - precision: 0.7102 - recall: 0.5764 - val_loss: 0.7980 - val_accuracy: 0.5992 - val_precision: 0.6450 - val_recall: 0.5199
Epoch 16/50
590/590 [==============================] - 1035s 2s/step - loss: 0.6981 - accuracy: 0.6717 - precision: 0.7155 - recall: 0.5966 - val_loss: 0.8167 - val_accuracy: 0.5951 - val_precision: 0.6297 - val_recall: 0.5237
12-08-21 02:50:28 - Saving model weights to /vserver/storages/crypto/crypto/.cache/weights/neuralnet.1.1.21.h5 ... done
Training evaluation: loss: 0.713 - accuracy: 0.65 - precision: 0.728 - recall: 0.51
Validation evaluation: loss: 0.78 - accuracy: 0.602 - precision: 0.662 - recall: 0.467
这是我的手动评估。
Evaluation:
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
version: 1.1.21
database: /etc/crypto/dataset/evaluation-alpaca-period_01-06-2021_01-07-2021_symbols_custom_1000/evaluation/
from: 01-06-21 16:00:00
till: 30-06-21 21:00:00
total days: 29.2
max profits bullish_1: 514.61%
max profits bearish_1: 406.51%
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Confusion matrix:
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
av_lost_profits av_profits av_win_profits earned lost precision profits recall usdt_profits win_ratio
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
normal 25%: 0.00 0.0 0.00 0.0 0.0 1.00 0.0% 0.86 10000.00 0.00
bullish_1 25%: -0.90 0.04% 0.94 3184.0 3209.0 0.03 4.56% 0.31 10448.88 0.50
bearish_1 25%: -0.97 -0.04% 0.87 5675.0 6060.0 0.03 -5.09% 0.57 9488.04 0.48
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
normal 35%: 0.00 0.0 0.00 0.0 0.0 1.00 0.0% 0.87 10000.00 0.00
bullish_1 35%: -0.90 0.04% 0.95 3139.0 3174.0 0.03 4.7% 0.31 10463.58 0.50
bearish_1 35%: -0.98 -0.04% 0.88 5636.0 6019.0 0.03 -5.18% 0.57 9479.89 0.48
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
normal 40%: 0.00 0.0 0.00 0.0 0.0 1.00 0.0% 0.88 10000.00 0.00
bullish_1 40%: -0.95 0.05% 1.00 2625.0 2644.0 0.04 6.03% 0.29 10601.21 0.50
bearish_1 40%: -1.01 -0.04% 0.92 5018.0 5355.0 0.03 -5.1% 0.56 9485.81 0.48
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
normal 50%: 0.00 0.0 0.00 0.0 0.0 1.00 0.0% 0.95 10000.00 0.00
bullish_1 50%: -1.23 0.1% 1.26 877.0 871.0 0.07 12.1% 0.16 11227.78 0.50
bearish_1 50%: -1.23 -0.06% 1.09 2260.0 2422.0 0.05 -8.13% 0.39 9190.03 0.48
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
normal 55%: 0.00 0.0 0.00 0.0 0.0 0.99 0.0% 0.98 10000.00 0.00
bullish_1 55%: -1.27 0.08% 1.31 400.0 398.0 0.07 9.83% 0.08 10948.53 0.50
bearish_1 55%: -1.26 -0.05% 1.09 998.0 1044.0 0.05 -6.16% 0.17 9360.44 0.49
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
normal 60%: 0.00 0.0 0.00 0.0 0.0 0.99 0.0% 0.99 10000.00 0.00
bullish_1 60%: -1.38 0.06% 1.48 171.0 178.0 0.09 5.81% 0.04 10508.35 0.49
bearish_1 60%: -1.26 -0.2% 1.10 286.0 355.0 0.04 -24.79% 0.05 7692.55 0.45
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
normal 65%: 0.00 0.0 0.00 0.0 0.0 0.99 0.0% 1.00 10000.00 0.00
bullish_1 65%: -1.47 0.21% 1.67 74.0 70.0 0.12 13.88% 0.03 11321.62 0.51
bearish_1 65%: -1.37 -0.02% 1.17 80.0 80.0 0.06 -1.21% 0.02 9618.41 0.50
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
normal 70%: 0.00 0.0 0.00 0.0 0.0 0.99 0.0% 1.00 10000.00 0.00
bullish_1 70%: -1.37 0.0% 1.73 30.0 36.0 0.11 0.09% 0.01 9930.77 0.45
bearish_1 70%: -1.72 0.33% 1.76 25.0 16.0 0.12 11.69% 0.01 10982.67 0.61
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
normal 75%: 0.00 0.0 0.00 0.0 0.0 0.99 0.0% 1.00 10000.00 0.00
bullish_1 75%: -1.08 0.48% 2.10 12.0 13.0 0.16 10.1% 0.01 11008.72 0.48
bearish_1 75%: -0.60 0.99% 2.58 5.0 5.0 0.10 9.91% 0.00 10995.70 0.50
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
normal 85%: 0.00 0.0 0.00 0.0 0.0 0.99 0.0% 1.0 10000.00 0.00
bullish_1 85%: -0.62 -0.14% 0.84 1.0 2.0 0.00 -0.41% 0.0 9958.04 0.33
bearish_1 85%: 0.00 0 0.00 0.0 0.0 0.00 0.0% 0.0 10000.00 0.00
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
normal 95%: 0.0 0.0 0.0 0.0 0.0 0.99 0.0% 1.0 10000.0 0.0
bullish_1 95%: 0.0 0.0 0.0 0.0 0.0 0.00 0.0% 0.0 10000.0 0.0
bearish_1 95%: 0.0 0.0 0.0 0.0 0.0 0.00 0.0% 0.0 10000.0 0.0
然而图像的处理方式相同（对于代码视图我关于图像处理Python 的另一个问题 - Keras - 这是相同的图像预处理吗？）
精密公式。
precision = true_positives / (true_positives + false_positives)
有谁知道可能导致这种情况的原因？