![]() After the entire scraping process is complete, all "final" errors will be printed as a JSON into a file called "finalErrors.json"(assuming you provided a logPath). If a request fails "indefinitely", it will be skipped. Number of repetitions depends on the global config option "maxRetries", which you pass to the Scraper. Nodejs-web-scraper will automatically repeat every failed request(except 404,400,403 and invalid images). In this case, all that needs to be done, is to use the usePuppeteer option. Let's say you have a site, whose pages perform some ajax requests, right after the DOM is loaded, with its initial html. Scrape site that loads additional content via ajax If you need to perform a highly customized task on a SPA site, that requires complex in-browser operations, you should learn to use Puppeteer directly. Using Puppeteer will just make the process much slower.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |