node.js - Unhandled promise rejection. This error originated either by throwing inside of an async function ... Node + Puppeteer

Question

I'm building a simple web scraper with puppeteer, that suppose to fetch from pexels.com image according to city name come from front-end req.

There are some cities that do not have a pic at the site, so I try to catch those cases by sending to the front-end the first pic in there suggestions to search.

While working on localhost, all work at expected, cities that have pic either not and { catch }, but when upload backend to Heroku, only cities such as London and Madrid got pic, but Tel-Aviv, etc.. no.

-----------------------$ heroku logs --tail

-11-27T09:09:19.340459+00:00 app[web.1]: (node:36) UnhandledPromiseRejectionWarning: Error: 
Evaluation failed: TypeError: Cannot read property 'getAttribute' of null
2019-11-27T09:09:19.340473+00:00 app[web.1]: at __puppeteer_evaluation_script__:4:11
2019-11-27T09:09:19.340476+00:00 app[web.1]: at ExecutionContext._evaluateInternal 
(/app/node_modules/puppeteer/lib/ExecutionContext.js:122:13)
2019-11-27T09:09:19.340479+00:00 app[web.1]: at processTicksAndRejections 
(internal/process/task_queues.js:93:5)
 2019-11-27T09:09:19.340481+00:00 app[web.1]: at async ExecutionContext.evaluate 
(/app/node_modules/puppeteer/lib/ExecutionContext.js:48:12)
2019-11-27T09:09:19.340483+00:00 app[web.1]: at async /app/controllers/webCrawler.js:13:17
2019-11-27T09:09:19.340486+00:00 app[web.1]: -- ASYNC --
2019-11-27T09:09:19.340489+00:00 app[web.1]: at ExecutionContext.<anonymous> 
(/app/node_modules/puppeteer/lib/helper.js:111:15)
2019-11-27T09:09:19.340490+00:00 app[web.1]: at DOMWorld.evaluate 
(/app/node_modules/puppeteer/lib/DOMWorld.js:112:20)
2019-11-27T09:09:19.340493+00:00 app[web.1]: at processTicksAndRejections 
(internal/process/task_queues.js:93:5)
2019-11-27T09:09:19.340496+00:00 app[web.1]: -- ASYNC --
2019-11-27T09:09:19.340498+00:00 app[web.1]: at Frame.<anonymous> 
(/app/node_modules/puppeteer/lib/helper.js:111:15)
2019-11-27T09:09:19.340500+00:00 app[web.1]: at Page.evaluate 
(/app/node_modules/puppeteer/lib/Page.js:863:43)
2019-11-27T09:09:19.340502+00:00 app[web.1]: at Page.<anonymous> 
(/app/node_modules/puppeteer/lib/helper.js:112:23)
2019-11-27T09:09:19.340504+00:00 app[web.1]: at /app/controllers/webCrawler.js:14:8
2019-11-27T09:09:19.340506+00:00 app[web.1]: at processTicksAndRejections 
(internal/process/task_queues.js:93:5)
2019-11-27T09:09:19.340590+00:00 app[web.1]: (node:36) UnhandledPromiseRejectionWarning: 
Unhandled promise rejection. This error originated either by throwing inside of an async 
function without a catch block, or by 
rejecting a promise which was not handled with 
.catch(). (rejection id: 3)

---

-----------------------------------my code

const puppeteer = require('puppeteer');

const handleScrape = (req, res) => {
 (async () => {
   const browser = await puppeteer.launch({
   args: [
    '--no-sandbox',
    '--disable-setuid-sandbox',
   ]
 });
const page = await browser.newPage();
await page.goto(
  `https://www.pexels.com/search/${req.body.name.replace(' ', '%20')}/`
);

const img = await page.evaluate(() => {
  try {
    return document
      .querySelector('a.js-photo-link img')
      .getAttribute('data-tiny-src');
  } catch {
    let url = document
      .querySelector('a.sponsored-photos__photo__link')
      .getAttribute('style');
    return url.slice(22, url.length - 1);
  }
});
res.json(img);
await browser.close();
})();
};

module.exports = {
  handleScrape
};

---

I did try to do the catch in few ways (with { then } and catch() and not catch{}, catch the whole evaluate, etc...) but for no good, and anyhow can't understand why it is working on local machine and not on deploy.

Thanks in advance :)

** Edit **

unlike UnhandledPromiseRejectionWarning: This error originated either by throwing inside of an async function without a catch block, I want to send a response in the catch method if there is no image gallery at that page I want to scrape over the suggestions, and as I mentioned at the post, the weird is all worked at developing mode, even have selenium as witness ;-)

标签： node.jspromiseasync-awaittry-catchpuppeteer

Answer 1

Always put your await calls of async function in try/catch block to avoid UnhandledPromiseRejection Error.

Here is short edit in you existing code :

const puppeteer = require('puppeteer');

const handleScrape = (req, res) => {
    (async () => {
        try{
            const browser = await puppeteer.launch({
                args: [
                    '--no-sandbox',
                    '--disable-setuid-sandbox',
                ]
            });
            const page = await browser.newPage();
            await page.goto(
                `https://www.pexels.com/search/${req.body.name.replace(' ', '%20')}/`
            );
    
            const img = await page.evaluate(() => {
                try {
                    return document
                        .querySelector('a.js-photo-link img')
                        .getAttribute('data-tiny-src');
                } catch {
                    let url = document
                        .querySelector('a.sponsored-photos__photo__link')
                        .getAttribute('style');
                    return url.slice(22, url.length - 1);
                }
            });
            res.json(img);
            await browser.close();
        } catch(e){
            console.log("Error Occurred", e)
        }
    })();
};

module.exports = {
    handleScrape
};