python - What is the best clean way to handle three datasets with different size for a training neural net?

Question

I have three datasets,each has different lengths. I have a model with three input pipeline (similar to Siamese, but three input pipeline) and therefore, I have to separately feed the data. I cannot concatenate them. The first one has around 600 ,second one has 3000 and last one has 40000 samples. In each training loop, I want to go through them and feed them into my model.

I defined three datasets and three dataloaders in pytorch; however, I am not sure how to iterate over them?

In the case of two, I have tried cycle over the smaller dataset and worked. But it is not working for three.

for step, ((images_Di, labels_Di),(images_Dj, labels_Dj)) in enumerate(zip(loader_D2, cycle(loader_D1))):

I appreciate if any one help me how is best way to handle them.

标签： pythondeep-learningdatasetpytorch

Answer 1

为什么要拥有 3 个不同的数据集/加载器？如果每个数据集中元素的大小相同，那么您只需将所有数据集中的图像和标签连接起来，然后创建一个 pytorch 数据集和加载器。那不是更简单吗？