tensorflow - how important is the loss difference between training and validation data at the beginning when training a neuronal network?

Question

Short question: Is the difference between validation and training loss at the beginning of the training (first epochs) a good indicator for the amount of data that should be used? E.g would it be a good method to increase the amount of data until the difference at the beginning is as small as possible? It would save me time and computation.

backround: I am working on a neuronal network that overfits very fast. The best result after applying many different techniques like dropouts, batch normalization, reducing learning rate, reducing batch size, increasing variety of data, reducing layers, increasing filter sizes ..... is still very bad. While the training loss decreases very well, validation loss overfits too early(with too early I mean, the desired loss is not reached, it should be many times less) Since the training with my dataset ~200 samples took 24 hours for 50 epochs, I was hoping to find a way to fight against overfitting with all the methods I described above, before increasing the amount of data. Because nothing helped I am at the point of increasing the amount of data. I am thinking about how much data could be enough for my network to eliminate overfitting. I know that this it is not easy to answer because it depends on the complexity of the data and the task I am trying to solve.. therefore I try to generalize my question to:

标签： tensorflowartificial-intelligenceconv-neural-networktraining-dataloss-function

Answer 1

简短问题的简短回答：否

解释：（train_loss - val_loss）与训练模型所需的数据量之间存在相关性，但还有许多其他因素可能是大数据的来源（train_loss - val_loss）。例如，您的网络架构太小，因此您的模型很快就会过拟合。或者，您的验证集不反映训练数据。或者你的学习率太大了。或者...

所以我的建议是：在另一个 SO 问题中阐述你的问题，然后问“我可能做错了什么？”