python - 多线程以在 Python 中更快地下载图像
问题描述
我有一个包含 400 万个汽车数据及其图片网址的列表。数据以下列方式存储在数据库中：
image_paths: ["1FMCU0GD8HUA54186_0.jpeg", "1FMCU0GD8HUA54186_1.jpeg", "1FMCU0GD8HUA54186_2.jpeg", "1FMCU0GD8HUA54186_3.jpeg", "1FMCU0GD8HUA54186_4.jpeg", "1FMCU0GD8HUA54186_5.jpeg", "1FMCU0GD8HUA54186_6.jpeg", "1FMCU0GD8HUA54186_7.jpeg", "1FMCU0GD8HUA54186_8.jpeg", "1FMCU0GD8HUA54186_9.jpeg", "1FMCU0GD8HUA54186_10.jpeg", "1FMCU0GD8HUA54186_11.jpeg", "1FMCU0GD8HUA54186_12.jpeg", "1FMCU0GD8HUA54186_13.jpeg", "1FMCU0GD8HUA54186_14.jpeg", "1FMCU0GD8HUA54186_15.jpeg", "1FMCU0GD8HUA54186_16.jpeg", "1FMCU0GD8HUA54186_17.jpeg", "1FMCU0GD8HUA54186_18.jpeg", "1FMCU0GD8HUA54186_19.jpeg"]}
image_urls: ['https://static.cargurus.com/images/forsale/2018/12/27/23/15/2017_ford_escape-pic-2184995423804160536-1024x768.jpeg', 'https://static.cargurus.com/images/forsale/2018/12/27/23/15/2017_ford_escape-pic-4540782425872667330-1024x768.jpeg', 'https://static.cargurus.com/images/forsale/2018/12/27/23/15/2017_ford_escape-pic-7243950089579631983-1024x768.jpeg', 'https://static.cargurus.com/images/forsale/2018/12/27/23/15/2017_ford_escape-pic-7710811720569469291-1024x768.jpeg', 'https://static.cargurus.com/images/forsale/2018/12/27/23/15/2017_ford_escape-pic-4031294549357151940-1024x768.jpeg', 'https://static.cargurus.com/images/forsale/2018/12/27/23/15/2017_ford_escape-pic-2789177086013778346-1024x768.jpeg', 'https://static.cargurus.com/images/forsale/2018/12/27/23/15/2017_ford_escape-pic-2720528876996226217-1024x768.jpeg', 'https://static.cargurus.com/images/forsale/2018/12/27/23/15/2017_ford_escape-pic-569475168753946872-1024x768.jpeg', 'https://static.cargurus.com/images/forsale/2018/12/27/23/15/2017_ford_escape-pic-72573478566280418-1024x768.jpeg', 'https://static.cargurus.com/images/forsale/2018/12/27/23/15/2017_ford_escape-pic-16994607167444151-1024x768.jpeg', 'https://static.cargurus.com/images/forsale/2018/12/27/23/15/2017_ford_escape-pic-8688248577169769701-1024x768.jpeg', 'https://static.cargurus.com/images/forsale/2018/12/27/23/15/2017_ford_escape-pic-2906095547280134476-1024x768.jpeg', 'https://static.cargurus.com/images/forsale/2018/12/27/23/15/2017_ford_escape-pic-4407739760966857172-1024x768.jpeg', 'https://static.cargurus.com/images/forsale/2018/12/27/23/15/2017_ford_escape-pic-3736759056303011298-1024x768.jpeg', 'https://static.cargurus.com/images/forsale/2018/12/27/23/15/2017_ford_escape-pic-4880108030264007082-1024x768.jpeg', 'https://static.cargurus.com/images/forsale/2018/12/27/23/15/2017_ford_escape-pic-267003030201578621-1024x768.jpeg', 'https://static.cargurus.com/images/forsale/2018/12/27/23/15/2017_ford_escape-pic-7577378734962701801-1024x768.jpeg', 'https://static.cargurus.com/images/forsale/2018/12/27/23/15/2017_ford_escape-pic-2130803651949430876-1024x768.jpeg', 'https://static.cargurus.com/images/forsale/2018/12/27/23/15/2017_ford_escape-pic-8952104950656887251-1024x768.jpeg', 'https://static.cargurus.com/images/forsale/2018/12/27/23/15/2017_ford_escape-pic-6495738051060732844-1024x768.jpeg']
对于 400 万辆汽车，我有这两列。现在我想尽可能快地下载图像，据我的研究，我知道多线程是做到这一点的最佳方式，对此我一无所知。任何人都可以帮我解决代码吗？我希望它在python中。为了下载图像，我已经编写了函数。
def downloadImageFunc(self,fileName,url):
# print url
resp1=urllib.urlopen(url)
self.uploadUrl(resp1,fileName)