python - 在 Selenium Python 中查找类名的所有元素

Question

我正在尝试从此页面https://www.centris.ca/en/properties~for-rent?view=Thumbnail提取所有广告，然后单击它们以提取数据。

出于某种原因，尽管使用了“find_elements”，但我只能提取第一个条目。

这是我的代码：

from selenium import webdriver
driver = webdriver.Chrome()
driver.get("https://www.centris.ca/en/properties~for-rent?view=Thumbnail")

list = driver.find_elements_by_xpath("//div[contains(@class, 'thumbnailItem col-12 col-sm-6 col-md-4 col-lg-3')]")
print(list)
for l in list:
    l.click()

使用 xpath （或其他方法）提取所有元素的正确方法是什么？

标签： pythonseleniumxpathcss-selectorswebdriverwait

Answer 1

要href从此页面https://www.centris.ca/en/properties~for-rent?view=Thumbnail提取所有广告的属性，您需要诱导WebDriverWait并且visibility_of_all_elements_located()您可以使用以下任一定位器策略：

• 使用CSS_SELECTOR：

  driver.get("https://www.centris.ca/en/properties~for-rent?view=Thumbnail")
  print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div[data-id='templateThumbnailItem'] a")))])
  

• 使用XPATH：

  driver.get("https://www.centris.ca/en/properties~for-rent?view=Thumbnail")
  print([my_elem.get_attribute("href") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[@data-id='templateThumbnailItem']//a")))])
  

• 控制台输出：

  ['https://www.centris.ca/en/lofts-studios~for-rent~quebec-la-cite-limoilou/22499659?view=Summary', 'https://www.centris.ca/en/lofts-studios~for-rent~quebec-la-cite-limoilou/22499659?view=Summary', 'https://www.centris.ca/en/lofts-studios~for-rent~quebec-la-cite-limoilou/22499659?view=Summary', 'https://www.centris.ca/en/lofts-studios~for-rent~quebec-la-cite-limoilou/22499659?view=Summary', 'https://www.centris.ca/en/lofts-studios~for-rent~quebec-la-cite-limoilou/22499659?view=Summary', None, 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/12049555?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/12049555?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/12049555?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/12049555?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/12049555?view=Summary', None, 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-le-sud-ouest/23940814?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-le-sud-ouest/23940814?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-le-sud-ouest/23940814?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-le-sud-ouest/23940814?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-le-sud-ouest/23940814?view=Summary', None, 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/27006991?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/27006991?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/27006991?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/27006991?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/27006991?view=Summary', None, 'https://www.centris.ca/en/condos-apartments~for-rent~levis-desjardins/19288399?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~levis-desjardins/19288399?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~levis-desjardins/19288399?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~levis-desjardins/19288399?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~levis-desjardins/19288399?view=Summary', None, 'https://www.centris.ca/en/condos-apartments~for-rent~mont-tremblant/17619353?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~mont-tremblant/17619353?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~mont-tremblant/17619353?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~mont-tremblant/17619353?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~mont-tremblant/17619353?view=Summary', None, 'https://www.centris.ca/en/condos-apartments~for-rent~quebec-sainte-foy-sillery-cap-rouge/21825148?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~quebec-sainte-foy-sillery-cap-rouge/21825148?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~quebec-sainte-foy-sillery-cap-rouge/21825148?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~quebec-sainte-foy-sillery-cap-rouge/21825148?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~quebec-sainte-foy-sillery-cap-rouge/21825148?view=Summary', None, 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/21424641?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/21424641?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/21424641?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/21424641?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/21424641?view=Summary', None, 'https://www.centris.ca/en/condos~for-sale~bromont/26233302?view=Summary', 'https://www.centris.ca/en/condos~for-sale~bromont/26233302?view=Summary', 'https://www.centris.ca/en/condos~for-sale~bromont/26233302?view=Summary', 'https://www.centris.ca/en/condos~for-sale~bromont/26233302?view=Summary', 'https://www.centris.ca/en/condos~for-sale~bromont/26233302?view=Summary', None, 'https://www.centris.ca/en/houses~for-rent~westmount/17444501?view=Summary', 'https://www.centris.ca/en/houses~for-rent~westmount/17444501?view=Summary', 'https://www.centris.ca/en/houses~for-rent~westmount/17444501?view=Summary', 'https://www.centris.ca/en/houses~for-rent~westmount/17444501?view=Summary', 'https://www.centris.ca/en/houses~for-rent~westmount/17444501?view=Summary', None, 'https://www.centris.ca/en/condos-apartments~for-rent~laval-chomedey/19011333?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~laval-chomedey/19011333?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~laval-chomedey/19011333?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~laval-chomedey/19011333?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~laval-chomedey/19011333?view=Summary', None, 'https://www.centris.ca/en/condos-apartments~for-rent~brossard/20981895?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~brossard/20981895?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~brossard/20981895?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~brossard/20981895?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~brossard/20981895?view=Summary', None, 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-le-sud-ouest/11477282?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-le-sud-ouest/11477282?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-le-sud-ouest/11477282?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-le-sud-ouest/11477282?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-le-sud-ouest/11477282?view=Summary', None, 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/12730891?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/12730891?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/12730891?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/12730891?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/12730891?view=Summary', None, 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/14530697?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/14530697?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/14530697?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/14530697?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/14530697?view=Summary', None, 'https://www.centris.ca/en/houses~for-rent~blainville/20565674?view=Summary', 'https://www.centris.ca/en/houses~for-rent~blainville/20565674?view=Summary', 'https://www.centris.ca/en/houses~for-rent~blainville/20565674?view=Summary', 'https://www.centris.ca/en/houses~for-rent~blainville/20565674?view=Summary', 'https://www.centris.ca/en/houses~for-rent~blainville/20565674?view=Summary', None, 'https://www.centris.ca/en/condos-apartments~for-rent~quebec-la-cite-limoilou/17234044?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~quebec-la-cite-limoilou/17234044?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~quebec-la-cite-limoilou/17234044?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~quebec-la-cite-limoilou/17234044?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~quebec-la-cite-limoilou/17234044?view=Summary', None, 'https://www.centris.ca/en/houses~for-sale~montreal-ville-marie/17988959?view=Summary', 'https://www.centris.ca/en/houses~for-sale~montreal-ville-marie/17988959?view=Summary', 'https://www.centris.ca/en/houses~for-sale~montreal-ville-marie/17988959?view=Summary', 'https://www.centris.ca/en/houses~for-sale~montreal-ville-marie/17988959?view=Summary', 'https://www.centris.ca/en/houses~for-sale~montreal-ville-marie/17988959?view=Summary', None, 'https://www.centris.ca/en/condos-apartments~for-rent~sainte-julie/14344430?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~sainte-julie/14344430?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~sainte-julie/14344430?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~sainte-julie/14344430?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~sainte-julie/14344430?view=Summary', None, 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/28728702?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/28728702?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/28728702?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/28728702?view=Summary', 'https://www.centris.ca/en/condos-apartments~for-rent~montreal-ville-marie/28728702?view=Summary', None]
  

• 注意：您必须添加以下导入：

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC