时间戳

首页 > 解决方案 > Node.js：如何通过 HTTPS/POST 请求下载文件

node.js - Node.js：如何通过 HTTPS/POST 请求下载文件

问题描述

我正在尝试从 www.borsaistanbul.com 下载文件 对于某些文件（例如链接下的文件=> https://www.borsaistanbul.com/veriler/verileralt/hisse-senetleri-piyasasi-verileri/bulten- verileri ) 他们提供了文件路径，所以我可以通过 https.get(downloadLink) 轻松下载它们。

但是对于https://www.borsaistanbul.com/veriler/verileralt/hisse-senetleri-piyasasi-verileri/piyasa-verileri下的文件，它们不提供路径和下载链接。我正在尝试下载名为“Üye Bazında Seanslık İşlem Sıralaması”（第 2 行的那个）我可能错了，但据我了解，当您单击旁边的下载图片时，您的浏览器会生成一个POST 请求，然后它在服务器端触发 smth，然后服务器将文件提供给您。我在 chromeDeveloper 工具的帮助下找到了 POST 请求并尝试模拟它，但它似乎不起作用。任何人都可以帮助并向我展示如何下载此文件的方法吗？

这是我尝试过的示例代码：

fs = require('fs');
const request = require('request');
/* Create an empty file where we can save data */
let file = fs.createWriteStream(`denemePost.zip`);
/* Using Promises so that we can use the ASYNC AWAIT syntax */        

new Promise((resolve, reject) => {
    let stream = request.post({
        /* Here you should specify the exact link to the file you are trying to download */
        uri: 'https://www.borsaistanbul.com/veriler/verileralt/hisse-senetleri-piyasasi-verileri/bulten-verileri',
        headers: {
            // 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
            'Accept' : 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
            'Accept-Encoding': 'gzip, deflate, br',
            // 'Accept-Language': 'en-US,en;q=0.9,fr;q=0.8,ro;q=0.7,ru;q=0.6,la;q=0.5,pt;q=0.4,de;q=0.3',
            'Accept-Language' : 'en-US,en;q=0.9',
            'Cache-Control': 'max-age=0',
            'Connection': 'keep-alive',
            'Content-Length' : '7511',
            'Content-Type' : 'application/x-www-form-urlencoded',
            'Cookie' : 'ASP.NET_SessionId=vugebk1zob2fw2hgxiftjg1z; cPER=!SmE/fvI1sjF1DqtSzYfA84hhMFmKdR+VmPTaX1WlhB8KHfkS3iP2fO2FK2iyUzwiDyupy85iZItfoeo=; _ga=GA1.2.534681471.1587587675; _gid=GA1.2.113108587.1588205109',
            'Host': 'www.borsaistanbul.com',
            'Origin' : 'null',
            'Sec-Fetch-Dest': 'document',
            'Sec-Fetch-Mode' : 'navigate', 
            'Sec-Fetch-Site' : 'same-origin',
            'Sec-Fetch-User': '?1',
            'Upgrade-Insecure-Requests': '1',
            // 'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36'
            'User-Agent' : 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.122 Safari/537.36'
        },
        /* GZIP true for most of the websites now, disable it if you don't need it */
        gzip: true
    })
    .pipe(file)
    .on('finish', () => {
        console.log(`The file is finished downloading.`);
        resolve();
    })
    .on('error', (error) => {
        reject(error);
    })
})
.catch(error => {
    console.log(`Something happened: ${error}`);
});

任何帮助将不胜感激，在此先感谢

标签： node.jshttpshttp-post

解决方案

如果有人试图完成类似的事情，我找到了一种解决方法。我已经下载了带有 puppeteer 库的文件。

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch({headless: false,slowMo: 250});
  const page = await browser.newPage();
  await page.goto('https://www.borsaistanbul.com/veriler/verileralt/hisse-senetleri-piyasasi-verileri/piyasa-verileri');
  page.once('load', () => console.log('Page loaded!'));
  await page.waitForSelector('#TextContent_C001_lbtnUyeBazindaGunlukIslemSiralamasi');
  await page.click('#TextContent_C001_lbtnUyeBazindaGunlukIslemSiralamasi');
  await browser.close();
})();

推荐阅读