python-3.x - 从 CSV 下载照片列表
问题描述
我尝试从 CSV 文件下载照片列表。我想保存链接列表列中的所有照片,并将其保存到与链接列表列同名的文件夹中。
预期的:
CSV 文件:
1 a-Qc01o78E a-rrBuci0w a-qj8s5nlM a-Cwciy2zx
0 2 https://photo.yupoo.com/ven-new/9f13389c/big.jpg https://photo.yupoo.com/ven-new/8852c424/big.jpg https://photo.yupoo.com/ven-new/650d84fd/big.jpg https://photo.yupoo.com/ven-new/a99f9e52/big.jpg
1 3 https://photo.yupoo.com/ven-new/f0adc019/big.jpg https://photo.yupoo.com/ven-new/c434624c/big.jpg https://photo.yupoo.com/ven-new/bed9125c/big.jpg https://photo.yupoo.com/ven-new/2d0b7a67/big.jpg
2 4 https://photo.yupoo.com/ven-new/8844627a/big.jpg https://photo.yupoo.com/ven-new/edda4ec4/big.jpg https://photo.yupoo.com/ven-new/3283fe57/big.jpg https://photo.yupoo.com/ven-new/1f6425e5/big.jpg
3 5 https://photo.yupoo.com/ven-new/eeb8b78a/big.jpg https://photo.yupoo.com/ven-new/6cdcbbf7/big.jpg https://photo.yupoo.com/ven-new/f64ca040/big.jpg https://uvd.yupoo.com/ven-new/22259049_oneTrue.jpg
4 6 https://photo.yupoo.com/ven-new/9c3e9a92/big.jpg https://photo.yupoo.com/ven-new/ea257725/big.jpg https://photo.yupoo.com/ven-new/64b5a57f/big.jpg https://uvd.yupoo.com/ven-new/22257899_oneTrue.jpg
5 7 https://photo.yupoo.com/ven-new/baaf8945/big.jpg https://photo.yupoo.com/ven-new/e9cc7392/big.jpg https://photo.yupoo.com/ven-new/753418b5/big.jpg https://uvd.yupoo.com/ven-new/22257619_oneTrue.jpg
6 8 https://photo.yupoo.com/ven-new/a325f44a/big.jpg https://photo.yupoo.com/ven-new/671be145/big.jpg https://photo.yupoo.com/ven-new/1742a09d/big.jpg https://photo.yupoo.com/ven-new/c9d0aa0f/big.jpg
7 9 https://photo.yupoo.com/ven-new/367cf72a/big.jpg https://photo.yupoo.com/ven-new/ae7f1b1b/big.jpg https://photo.yupoo.com/ven-new/d0ef54ed/big.jpg https://photo.yupoo.com/ven-new/2d0905df/big.jpg
8 10 https://photo.yupoo.com/ven-new/3fcacff3/big.jpg https://photo.yupoo.com/ven-new/c4ea9b1e/big.jpg https://photo.yupoo.com/ven-new/683db958/big.jpg https://photo.yupoo.com/ven-new/3b065995/big.jpg
9 11 https://photo.yupoo.com/ven-new/c7de704a/big.jpg https://photo.yupoo.com/ven-new/92abc9ea/big.jpg https://photo.yupoo.com/ven-new/bd1083db/big.jpg https://photo.yupoo.com/ven-new/a9086d26/big.jpg
10 12 https://photo.yupoo.com/ven-new/fc481727/big.jpg https://photo.yupoo.com/ven-new/a49c94df/big.jpg
11 13 https://photo.yupoo.com/ven-new/1e0e0e10/big.jpg https://photo.yupoo.com/ven-new/62580909/big.jpg
12 14 https://photo.yupoo.com/ven-new/934b423e/big.jpg https://photo.yupoo.com/ven-new/74b81853/big.jpg
13 15 https://photo.yupoo.com/ven-new/adf878b2/big.jpg https://photo.yupoo.com/ven-new/5ad881c3/big.jpg
14 16 https://photo.yupoo.com/ven-new/59dc1203/big.jpg https://photo.yupoo.com/ven-new/3cd676ac/big.jpg
15 17 https://photo.yupoo.com/ven-new/6d8eb080/big.jpg
16 18 https://photo.yupoo.com/ven-new/9a027ada/big.jpg
17 19 https://photo.yupoo.com/ven-new/bdeaf1b5/big.jpg
18 20 https://photo.yupoo.com/ven-new/1f293683/big.jpg
def get_photos(x):
with requests.Session() as c:
df = pd.read_csv("C:\\Users\\Lukasz\\Desktop\\PROJEKTY PYTHON\\W TRAKCIE\\YUOPOO SCRAPER\\FF_data_frame.csv")
NazwyKolum = pd.read_csv("C:\\Users\\Lukasz\\Desktop\\PROJEKTY PYTHON\\W TRAKCIE\\bf3_strona.csv")
c.get('https://photo.yupoo.com/')
c.headers.update({'referer': 'https://photo.yupoo.com/'})
URLS=(df[NazwyKolum['LINKS'][x]]) #.to_string(index=False))
print(URLS) #prints a list of links. Example https://photo.yupoo.com/ven-new/650d84fd/big.jpg, https://photo.yupoo.com/ven-new/bed9125c/big.jpg
#proxies = {'https': 'http://45.77.76.254:8080'}
res = c.get(URLS,timeout=None)
if res.status_code == 200:
return res.content
try:
for x in range(2,50):
with open("C:\\Users\\Lukasz\\Desktop\\PROJEKTY PYTHON\\W TRAKCIE\\YUOPOO SCRAPER\\"+str(x)+'NAZWAZDECIA.jpg', 'wb') as f:
f.write(get_photos(x))
except:
print("wr")
老实说,我已经不怎么处理它了,我浪费了很多时间,没有任何进展。谢谢你们。
解决方案
import pandas as pd
import requests, os
df = pd.read_csv("a.csv")
def create_directory(directory):
if not os.path.exists(directory):
os.makedirs(directory)
def download_save(url, folder):
create_directory(folder)
res = requests.get(url)
with open(f'{folder}/{url.split("/")[-2]}.jpg', 'wb') as f:
f.write(res.content)
for col in df.columns:
print(col)
for url in df[col].tolist():
print(url)
if str(url).startswith("http"):
download_save(url, col)
上面的代码将列名创建为目录并下载图像并将其以唯一名称保存在 url 中。
如下a.csv
:
a-Qc01o78E,a-rrBuci0w,a-qj8s5nlM,a-Cwciy2zx
https://photo.yupoo.com/ven-new/9f13389c/big.jpg,https://photo.yupoo.com/ven-new/8852c424/big.jpg,https://photo.yupoo.com/ven-new/650d84fd/big.jpg,https://photo.yupoo.com/ven-new/a99f9e52/big.jpg
https://photo.yupoo.com/ven-new/f0adc019/big.jpg,https://photo.yupoo.com/ven-new/c434624c/big.jpg,https://photo.yupoo.com/ven-new/bed9125c/big.jpg,https://photo.yupoo.com/ven-new/2d0b7a67/big.jpg
https://photo.yupoo.com/ven-new/8844627a/big.jpg,https://photo.yupoo.com/ven-new/edda4ec4/big.jpg,https://photo.yupoo.com/ven-new/3283fe57/big.jpg,https://photo.yupoo.com/ven-new/1f6425e5/big.jpg
https://photo.yupoo.com/ven-new/eeb8b78a/big.jpg,https://photo.yupoo.com/ven-new/6cdcbbf7/big.jpg,https://photo.yupoo.com/ven-new/f64ca040/big.jpg,https://uvd.yupoo.com/ven-new/22259049_oneTrue.jpg
https://photo.yupoo.com/ven-new/9c3e9a92/big.jpg,https://photo.yupoo.com/ven-new/ea257725/big.jpg,https://photo.yupoo.com/ven-new/64b5a57f/big.jpg,https://uvd.yupoo.com/ven-new/22257899_oneTrue.jpg
推荐阅读
- java - 如何在 Rest 客户端中使用自定义 java 对象作为请求参数?
- facebook-instant-articles - 帖子上的 Facebook Instant Article 链接
- java - Array/ArrayList 使用扫描仪输入进行排序
- html - Angular URL GET 不执行
- javascript - 如何在购物车下拉 JavaScript 中仅显示三个项目?
- .net - System.Runtime.Remoting.RemotingException:“未找到请求的服务”
- android - 我无法在 Alldocube iPlay 20 上安装 Android Treble
- javascript - 检查 `RestApi` 的渲染方法
- python - 单个烧瓶应用程序中多个子域的单独登录会话
- java - 在css html中使用冒号从多个div类中获取元素