时间戳

首页 > 解决方案 > 动态网页抓取 - 无法获取所有项目

python - 动态网页抓取 - 无法获取所有项目

问题描述

这是我正在测试的链接。

https://www.tokopedia.com/search?navsource=home&page=2&q=ipad&st=product'

我添加了一个滚动控制脚本，因为它看起来像一个动态页面。

while True:

    driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
    time.sleep(SCROLL_PAUSE_SEC)
    new_height = driver.execute_script("return document.body.scrollHeight")
    if new_height == last_height:
        break
    last_height = new_height

但是我仍然没有得到所有的项目。搜索词“ipad”列出了实际的 80 个项目。

这是我的测试代码。

import time
from latest_user_agents import get_random_user_agent
from selenium import webdriver
from selenium.common import exceptions
from selenium.webdriver.common.action_chains import ActionChains


def open_webdriver():
    options = webdriver.ChromeOptions()
    options.add_argument('--disable-dev-shm-usage')
    options.add_argument('--no-sandbox')
    options.add_argument('disable-gpu')
    options.add_argument('--ignore-certificate-errors')
    # options.add_argument('headless')
    options.add_experimental_option('excludeSwitches', ['enable-logging'])
    options.add_argument('--lang=en')
    user_agent = get_random_user_agent()
    print(user_agent)
    options.add_argument(f'user-agent={user_agent}')
    driver = webdriver.Chrome(r'c:\chromedriver\chromedriver.exe', options=options)
    return driver


driver = open_webdriver()
url = 'https://www.tokopedia.com/search?navsource=home&page=2&q=ipad&st=product'
driver.get(url)
time.sleep(1)
action = ActionChains(driver)
last_height = driver.execute_script("return document.body.scrollHeight")
SCROLL_PAUSE_SEC = 1
while True:

    driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
    time.sleep(SCROLL_PAUSE_SEC)
    new_height = driver.execute_script("return document.body.scrollHeight")
    if new_height == last_height:
        break
    last_height = new_height
items = driver.find_elements_by_xpath('//*[@id="zeus-root"]/div/div[2]/div/div[2]/div[3]/div[1]/div')
print('items',len(items))

for item in items:

    try:
        name = item.find_element_by_xpath('.//div/div/div/div/div/div[2]/a/div').text
        print(name)
    except exceptions.NoSuchElementException:
        print()

结果 在此处输入图像描述

我怎样才能得到所有的物品？

标签： pythonweb-scraping

解决方案

你的输出是正确的。该网站每页显示 80 个项目，但它们已为每页动态分页 15 个项目。实际上，我们无法修复一种技术来抓取网站，而不是网站向我们展示了我们必须使用什么技术来抓取网站。我想从后门抓取该网站，但没有获得许可。

我还将 selenium 与 bs4、scrapy-seleniumRequest、selenium 与 crawlSpider 一起使用，但它们都没有工作，并且分页非常复杂，这就是为什么我将 scrapy 与 selenium 一起使用并在 start_url 中进行分页，您可以随时增加或减少页码的范围. 这里我从 url 的起始页 2 到 10 进行分页，意思是总共 8 页，total items:120

代码：

import scrapy
from selenium import webdriver
from shutil import which
from scrapy.selector import Selector
from selenium.common import exceptions
from time import sleep

class ProductSpider(scrapy.Spider):
    name = "tok"
    allowed_domains = ['www.tokopedia.com']
    start_urls = ['https://www.tokopedia.com/search?navsource=home&page='+str(x)+'&q=ipad&st=product' for x in range(2,10)]

    def __init__(self):
        chrome_path = which("chromedriver")
        self.driver = webdriver.Chrome(executable_path=chrome_path)
        self.driver.set_window_size(1920, 1080)

    def parse(self, response):
        self.driver.get(response.url)
        sleep(5)

        try:
            
            cards = Selector(text=self.driver.page_source)
            items = cards.xpath('//*[@id="zeus-root"]/div/div[2]/div/div[2]/div[3]/div[1]/div')
            for item in items:
                yield{
                    'Name': item.xpath('.//div/div/div/div/div/div[2]/a/div/text()').get()
                   
                }
                
        except exceptions.NoSuchElementException:
            print('g')

输出：总输出的一部分。

{'Name': 'Gebyar Diskon'}
2021-08-03 09:12:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=2&q=ipad&st=product>
{'Name': 'iPad Pro 2021 M1 11" 2TB 1TB 512GB 256GB 128GB 11 inch 5th Gen'}
2021-08-03 09:12:47 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=2&q=ipad&st=product>
{'Name': 'Sisa 10'}
2021-08-03 09:12:47 [selenium.webdriver.remote.remote_connection] DEBUG: POST http://127.0.0.1:63786/session/1e6d1cb95aa9b366f82271bf502e49ef/url {"url": "https://www.tokopedia.com/search?navsource=home&page=8&q=ipad&st=product"}   
2021-08-03 09:12:51 [urllib3.connectionpool] DEBUG: http://127.0.0.1:63786 "POST /session/1e6d1cb95aa9b366f82271bf502e49ef/url HTTP/1.1" 200 14
2021-08-03 09:12:51 [selenium.webdriver.remote.remote_connection] DEBUG: Finished Request
2021-08-03 09:12:56 [selenium.webdriver.remote.remote_connection] DEBUG: GET http://127.0.0.1:63786/session/1e6d1cb95aa9b366f82271bf502e49ef/source {}
2021-08-03 09:12:56 [urllib3.connectionpool] DEBUG: http://127.0.0.1:63786 "GET /session/1e6d1cb95aa9b366f82271bf502e49ef/source HTTP/1.1" 200 156307
2021-08-03 09:12:56 [selenium.webdriver.remote.remote_connection] DEBUG: Finished Request
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=8&q=ipad&st=product>
{'Name': 'Apple iPad Air 4 64GB 256GB Wifi Only IPAD AIR 2020 - Garansi 1 Tahun - SpaceGray WIFI, 64GB ONLY'}       
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=4&q=ipad&st=product>
{'Name': 'Apple iPad Air 4 2020 64GB 10.9" WiFi Cellular 64 GRAY GREEN ROSE GOLD'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=4&q=ipad&st=product>
{'Name': 'Gebyar Diskon'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=4&q=ipad&st=product>
{'Name': 'Bisa TukarTambah'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=4&q=ipad&st=product>
{'Name': 'Bisa TukarTambah'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=4&q=ipad&st=product>
{'Name': 'Sisa 3'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=4&q=ipad&st=product>
{'Name': 'Bisa TukarTambah'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=6&q=ipad&st=product>
{'Name': 'iPad Mini 1 Wifi 16GB Original - Second Like New'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=6&q=ipad&st=product>
{'Name': 'RESMI Apple iPad Pro M1 12.9" 2021 128GB 256GB 512GB 1TB 2TB WIFI CELL'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=6&q=ipad&st=product>
{'Name': 'Apple iPad Air 4 2020 256GB 64GB Wifi Only 10.9" - BNIB Original'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=6&q=ipad&st=product>
{'Name': 'Sisa 2'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=6&q=ipad&st=product>
{'Name': 'Bisa TukarTambah'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=3&q=ipad&st=product>
{'Name': 'New iPad Pro M1 2021 Wi-Fi 256GB 11" 12.9" WiFi 256 GB'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=3&q=ipad&st=product>
{'Name': 'Apple iPad 8 2020 10.2" 128GB 32GB WiFi Cellular GRAY GREY GOLD SILVER'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=3&q=ipad&st=product>
{'Name': 'Bisa TukarTambah'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=3&q=ipad&st=product>
{'Name': 'Bisa TukarTambah'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=3&q=ipad&st=product>
{'Name': 'Gebyar Diskon'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=5&q=ipad&st=product>
{'Name': 'iPad Air 32GB Wifi Cell 4G LTE Retina Fullset EX RESMI APPLE'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=5&q=ipad&st=product>
{'Name': 'Bisa TukarTambah'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=5&q=ipad&st=product>
{'Name': 'LOGITECH FOLIO TOUCH iPad Air 4 2020 LOGITECH FOLIO TOUCH iPad Pro 11'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=5&q=ipad&st=product>
{'Name': 'Apple iPad 8 2020 10.2" 32GB 32 Gb WiFi Tablet'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=7&q=ipad&st=product>
{'Name': 'Apple iPad Air 4/ 4th Generation 2020 10.9 Inch 64GB Wifi Only'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=7&q=ipad&st=product>
{'Name': 'Bisa TukarTambah'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=7&q=ipad&st=product>
{'Name': 'Apple iPad 8 / 8th Gen 2020 10.2 Inch 128GB Wifi only BNIB'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=9&q=ipad&st=product>
{'Name': 'Apple iPad 8 10.2" Gen 8 2020 Wifi / Cellular Space Grey Silver Gold'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=9&q=ipad&st=product>
{'Name': 'iPad Mini 5 2019 256GB WiFi Cellular Garansi 1 Tahun 64gb - 256gb'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=9&q=ipad&st=product>
{'Name': 'iPad Pro 2020 12.9” 1TB 512GB 256GB 128GB 12.9 inc 4th Gen WIFI CELL'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=9&q=ipad&st=product>
{'Name': 'Sisa 2'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=9&q=ipad&st=product>
{'Name': 'Apple iPad Pro 3rd Gen 2018 12" inch 64gb cell + smartkeyboard folio'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=2&q=ipad&st=product>
{'Name': 'Bisa TukarTambah'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=8&q=ipad&st=product>
{'Name': 'Bisa TukarTambah'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=8&q=ipad&st=product>
{'Name': '[RESMI] Apple iPad Pro M1 12.9" 2021 128GB Wifi Cellular Gray Silver - Wifi Only, Gray Inter'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=8&q=ipad&st=product>
{'Name': 'Bisa TukarTambah'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=8&q=ipad&st=product>
{'Name': 'Bisa TukarTambah'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=4&q=ipad&st=product>
{'Name': 'Apple iPad Pro 3rd Gen 2018 11" inch Wifi Only 256GB Gray Silver'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=4&q=ipad&st=product>
{'Name': 'Gebyar Diskon'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=4&q=ipad&st=product>
{'Name': 'iPad Air 4 2020 64GB 256GB 10.9" Wifi Cell bisa Apple Pencil 64 256 GB'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=4&q=ipad&st=product>
{'Name': 'Gebyar Diskon'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=6&q=ipad&st=product>
{'Name': 'Bisa TukarTambah'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=6&q=ipad&st=product>
{'Name': 'Apple iPad Pro 2020 12.9" 1TB 512GB 256GB 128GB 12.9 inch 4th Gen WIFI'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=6&q=ipad&st=product>
{'Name': 'Bisa TukarTambah'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=3&q=ipad&st=product>
{'Name': 'Bisa TukarTambah'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=3&q=ipad&st=product>
{'Name': 'READY New iPad 9.7" iPad 2018 iPad 6 6th Gen Air 4 128gb Wifi Only'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=3&q=ipad&st=product>
{'Name': 'Gebyar Diskon'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=3&q=ipad&st=product>
{'Name': 'Sisa 2'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=5&q=ipad&st=product>
{'Name': '(IBOX) iPad Air 4 2020 64GB 256GB Wifi Cellular Garansi Resmi iBox'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=5&q=ipad&st=product>
{'Name': 'Bisa TukarTambah'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=5&q=ipad&st=product>
{'Name': 'Apple iPad PRO M1 2021 12.9" WiFi ONLY 2TB 1TB 512GB 256GB 128GB Resmi'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=5&q=ipad&st=product>
{'Name': 'IPAD 7 2019 10.2" 2019 WIFI 32 GB 128 GB'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=5&q=ipad&st=product>
{'Name': '(RESMI) iPad 8 2020 10.2" inch 128GB 128 WIFI Cellular Cell IBOX'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=5&q=ipad&st=product>
{'Name': 'Bisa TukarTambah'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=7&q=ipad&st=product>
{'Name': 'New iPad Pro M1 Chip 2021 Gen 5th 11 Inch 128GB 256GB 512GB 1TB 2TB'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=7&q=ipad&st=product>
{'Name': 'ipad 2019 128gb'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=7&q=ipad&st=product>
{'Name': 'Apple iPad PRO M1 2021 11" WiFi CELLULAR 2TB 1TB 512GB 256GB Resmi'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=7&q=ipad&st=product>
{'Name': '⭐⭐⭐⭐⭐ PROMO DISKON iPad 2 Wifi 3G 16GB Black - MC773ID/A'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=7&q=ipad&st=product>
{'Name': 'ipad pro (11-inch) 2018 64gb 64 cellular / wifi + case premium second'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=9&q=ipad&st=product>
{'Name': 'iPad Pro 2020 11” 1TB 512GB 256GB 128GB 11 inch 4th Gen WIFI CELL'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=9&q=ipad&st=product>
{'Name': 'Bisa TukarTambah'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=8&q=ipad&st=product>
{'Name': 'Apple iPad Pro 12.9 2021 XDR 128GB 256GB 512GB 1TB Wifi Only'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=8&q=ipad&st=product>
{'Name': 'Bisa TukarTambah'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=8&q=ipad&st=product>
{'Name': 'APPLE IPAD 7 128GB WIFI (2019)'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=8&q=ipad&st=product>
{'Name': 'iPad PRO 10.5 2017 64GB WIFI + Apple Pencil'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=8&q=ipad&st=product>
{'Name': 'Ipad 4th Gen Wifi 16GB Original Second Mulus Seperti Baru'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=6&q=ipad&st=product>
{'Name': 'Sisa 3'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=7&q=ipad&st=product>
{'Name': 'Apple iPad 8 2020 10.2" 128GB 32GB WiFi Cellular GRAY GREY GOLD SILVER'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=8&q=ipad&st=product>
{'Name': 'Apple iPad 8 2020 10.2" Inch 32GB WiFi Cellular GRAY GREY GOLD SILVER'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=8&q=ipad&st=product>
{'Name': 'Apple iPad Pro 2021 12.9 inch M1 Chip 128GB 256GB 512GB Wifi Only'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=8&q=ipad&st=product>
{'Name': 'Apple iPad Air 4 / 4th Gen 2020 10.9 Inch 256GB Wifi Only BNIB'}
2021-08-03 09:12:56 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=8&q=ipad&st=product>
{'Name': 'Bisa TukarTambah'}
2021-08-03 09:12:57 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=7&q=ipad&st=product>
{'Name': '[RESMI] Apple iPad Air 4 2020 256GB 64GB WIFI & CELL Garansi 1 Tahun'}
2021-08-03 09:12:57 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.tokopedia.com/search?navsource=home&page=8&q=ipad&st=product>
{'Name': 'Apple iPad 8 8th 2020 10.2" Wifi Cellular 128GB 32GB Grey Silver Gold'}
2021-08-03 09:12:57 [scrapy.core.engine] INFO: Closing spider (finished)
2021-08-03 09:12:57 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 2752,
 'downloader/request_count': 8,
 'downloader/request_method_count/GET': 8,
 'downloader/response_bytes': 749403,
 'downloader/response_count': 8,
 'downloader/response_status_count/200': 8,
 'elapsed_time_seconds': 75.359703,
 'finish_reason': 'finished',
 'finish_time': datetime.datetime(2021, 8, 3, 3, 12, 57, 16784),
 'httpcompression/response_bytes': 7774931,
 'httpcompression/response_count': 8,
 'item_scraped_count': 120

推荐阅读