python - 触发 Selenium 点击功能后无法获取标签文本
问题描述
http://211.21.63.217:81/ticket_online.aspx
我爬取网站电影列表数据,可以成功获取电影名称、电影时间。
然后我想在明天<span />
的电影时间抓取每部电影,所以我明天使用硒点击。
我确保 Selenium 也单击了<Span />
我设置time.sleep
的每个,并检查标签根是否正确,但明天movieTime
的结果是空的。
我不知道为什么。
这是我的代码:
# -*- coding: utf-8 -*-
import scrapy
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
from scrapy.http import HtmlResponse
from datetime import datetime
from datetime import timedelta
import time
# like 07/02
tomorrowDate = datetime.now() + timedelta(days=1)
tomorrow = tomorrowDate.strftime('%m/%d')
afterTomorrowDate = datetime.now() + timedelta(days=2)
afterTomorrow = afterTomorrowDate.strftime('%m/%d')
print(tomorrow)
print(afterTomorrow)
class CenturyasiaxinbeiSpider(scrapy.Spider):
name = 'CenturyasiaXinbei'
allowed_domains = ['www.centuryasia.com.tw', '211.21.63.217:81']
start_urls = ['http://211.21.63.217:81/ticket_online.aspx']
def __init__(self):
print('__init__')
self.driver = webdriver.Chrome('/Users/motogod19/chromedriver')
self.driver.get('http://211.21.63.217:81/ticket_online.aspx')
def parse(self, response):
firstHtmlNodes = response.xpath('//article[@id="ContentPlaceHolder1_Time_box"]//section[@class="tickets_movie_time_box"]')
for firstHtmlNode in firstHtmlNodes:
# print(firstHtmlNode)
# Today
movieCnName = firstHtmlNode.xpath('./div[@class="tickets_movie_time"]/div[@class="tickets_times"]/div[@class="tickets_times_t"]/div[@class="times_title"]/text()').get()
movieEnName = firstHtmlNode.xpath('./div[@class="tickets_movie_time"]/div[@class="tickets_times"]/div[@class="tickets_times_t"]/div[@class="times_title_en"]/text()').get()
movieTime = firstHtmlNode.xpath('./div[@class="tickets_movie_time"]/div[@class="tickets_times"]/div[@class="tickets_date"]/div[3]/div/div[@class="tickets_date_csbox"]/div/ul/li/text()').extract()
print(movieCnName)
print(movieEnName)
# sort the movie time
print(sorted(movieTime))
print('\n')
time.sleep(2)
allSpanDate = self.driver.find_elements_by_xpath("//span[text()='{}']".format(tomorrow))
for spanDate in allSpanDate:
ActionChains(self.driver).move_to_element(spanDate).perform()
spanDate.click()
time.sleep(1)
tomorrowResponse = HtmlResponse(url=self.driver.current_url, body=self.driver.page_source, encoding='utf-8')
secondHtmlNodes = tomorrowResponse.xpath('//article[@id="ContentPlaceHolder1_Time_box"]//section[@class="tickets_movie_time_box"]')
for secondHtmlNode in secondHtmlNodes:
# Tomorrow
movieCnName = secondHtmlNode.xpath('./div[@class="tickets_movie_time"]/div[@class="tickets_times"]/div[@class="tickets_times_t"]/div[@class="times_title"]/text()').get()
movieEnName = secondHtmlNode.xpath('./div[@class="tickets_movie_time"]/div[@class="tickets_times"]/div[@class="tickets_times_t"]/div[@class="times_title_en"]/text()').get()
# movieTime = secondHtmlNode.xpath('./div[@class="tickets_movie_time"]/div[@class="tickets_times"]/div[@class="tickets_date"]/div[3]/div/div[@class="tickets_date_csbox"]/div/ul/li/text()').extract()
movieTime = secondHtmlNode.xpath('./div[@class="tickets_movie_time"]/div[@class="tickets_times"]/div[@class="tickets_date"]/div[3]/div[@class="tickets_date_cts"]/ul')
# tickets_date_cts
print(movieCnName)
print(movieEnName)
# sort movie time
print(sorted(movieTime)) # Empty is here
print('\n')
time.sleep(60)
解决方案
试试这个 xpath,它适用于movieTime = firstHtmlNode.xpath()
和movieTime = secondHtmlNode.xpath
.
.//div[@class='tickets_date_cts']/ul/li
推荐阅读
- python - 模型中的“更新”会导致覆盖
- c - 冒泡排序与 C 中的结构
- php - Substr 数组中的所有值
- c# - 视图模型更新时视图不更新
- rest - 如何使用 HTTP API 将数据上传到 IBM IoT Platform 和 Cloudant 数据库?
- python - Django 测试客户端中的 UNIQUE 约束失败
- mysql - 引导日期选择器自定义格式保存到 DATE MySQL
- html - chrome和firefox浏览器输入中占位符大小的差异
- c# - C#。iText7. 在 pdf 中查找表格
- javascript - 在重新加载完整图表之前,highcharts 是否有回调函数(这不应该包括图例禁用)?