r - 使用 rvest 抓取网页时无法获得结果
问题描述
我想从VISA ATM 定位器获取西班牙 ATM 的位置。西班牙的结果显示在表格中,但我不知道如何访问这些元素。我努力了:
link <- "https://www.visa.com/atmlocator/mobile/index.jsp#(page:results,params:(query:Spain))"
visa_webpage <- read_html(link)
visa_webpage %>%
html_nodes("visaATMResultListItem") %>%
html_text()
解决方案
数据是从浏览器发出的 jquery 请求中动态检索的。您可以使用它的简化版本来检索数据
library(httr)
library(stringr)
library(jsonlite)
headers = c('User-Agent' = 'Mozilla/5.0')
params = list(
'callback' = 'jQuery112403101782845756018_1577837576284',
'request' = '{"wsRequestHeaderV2":{"requestTs":"","applicationId":"VATMLOC","requestMessageId":"test12345678","userId":"CDISIUserID","userBid":"10000108","correlationId":"909420141104053819418"},"requestData":{"culture":"en-US","distance":"60","distanceUnit":"mi","metaDataOptions":0,"location":{"address":null,"placeName":"Spain","geocodes":{"latitude":"40.227949660000036","longitude":"-3.6460631049999392"}},"options":{"sort":{"primary":"distance","direction":"asc"},"range":{"start":0,"count":8},"operationName":"and","findFilters":[{"filterName":"PLACE_NAME","filterValue":""},{"filterName":"CARD_ACCEPT","filterValue":""},{"filterName":"OPER_HRS","filterValue":""},{"filterName":"AIRPORT_CD","filterValue":""},{"filterName":"WHEELCHAIR","filterValue":""},{"filterName":"BRAILLE_AUDIO","filterValue":""},{"filterName":"BALANCE_INQUIRY","filterValue":""},{"filterName":"CHIP_CAPABLE","filterValue":""},{"filterName":"PIN_CHANGE","filterValue":""},{"filterName":"RESTRICTED","filterValue":""},{"filterName":"PLUS_ALLIANCE_NO_SURCHARGE_FEE","filterValue":""},{"filterName":"ACCEPTS_PLUS_SHARED_DEPOSIT","filterValue":""},{"filterName":"V_PAY_CAPABLE","filterValue":""},{"filterName":"READY_LINK","filterValue":""}],"useFirstAmbiguous":true}}}',
'_' = '1577837576288'
)
r <- httr::GET(url = 'https://www.visa.com/atmlocator_services/rest/findNearByATMs', httr::add_headers(.headers=headers), query = params)
data <- jsonlite::fromJSON(str_match(r%>%toString() , 'jQuery112403101782845756018_1577837576284\\((.*)\\)' )[1,2])
locations <- data.frame(data$responseData$foundATMLocations[1])
推荐阅读
- android - Java 或 Kotlin 文件“MainApplication.java”不包含包声明
- json - Groovy 从嵌套的 json 数组中收集值
- python-3.x - Django,无法在本地启动:主机
- amazon-web-services - terraform 启动和配置 EC2 AD
- linux-kernel - linux内核网络驱动编译失败
- python - 如何在 FiPy 中添加潜热项?
- drupal - 导入 Migrate 模块时出现 Sourceid1 错误
- bootstrap-5 - 使用 bootstrap 5 导航栏,我们如何让左对齐的项目在右下方溢出?
- amazon-redshift - Redshift:配置 SORTKEY AUTO 时 SVV_TABLE_INFO 中的空排序键统计信息
- algorithm - 编码算法如何处理单独的位块