r - r 令牌中的 HTTR POST
问题描述
我想在这个网站上抓取代理的数据:https ://thep.hoaphat.com.vn/distribution-systems
我正在使用以下代码,但得到“令牌不匹配错误”:
httr::POST(
url = "https://thep.hoaphat.com.vn/ajax/load_agency",
body = list(
type = "web",
product_id = "7",
province_id = "10",
member_type = "1"
),
encode = "form"
) -> res
dat <- httr::content(res)
str(dat)
我该如何解决?
解决方案
数据被请求到https://thep.hoaphat.com.vn/ajax/load_agency
做一点逆向工程,我发现请求是从这个 JS 文件发送的: https ://thep.hoaphat.com.vn/assets/js/functions.min.js
看看这个函数:
function loadAjax(t, e, n) {
var a = {
beforeSend: function() {},
success: function() {}
};
$.extend(a, n),
$.ajax({
headers: {
"X-CSRF-TOKEN": $('meta[name="csrf-token"]').attr("content")
},
type: "POST",
url: t,
data: e,
beforeSend: function() {
$("#loading_box").css({
visibility: "visible",
opacity: 0
}).animate({
opacity: 1
}, 200),
a.beforeSend()
},
success: function(t) {
$("#loading_box").animate({
opacity: 0
}, 200, function() {
$("#loading_box").css("visibility", "hidden")
}),
a.success(t)
},
error: function(t) {
$("#loading_box").animate({
opacity: 0
}, 200, function() {
$("#loading_box").css("visibility", "hidden")
}),
alert("Có lỗi xảy ra!")
}
})
}
您可以看到令牌来自meta
带有 name 的标签csrf-token
。现在您可以抓取该令牌值并发送请求以获取数据:
library(rvest)
pg <- html_session("https://thep.hoaphat.com.vn/distribution-systems")
token <- read_html(pg) %>%
html_node(xpath = "//meta[@name='csrf-token']") %>% html_attr("content")
pg <-
pg %>% rvest:::request_POST(
"https://thep.hoaphat.com.vn/ajax/load_agency",
config = httr::add_headers(`x-csrf-token` = token),
body = list(
type = "web",
product_id = 1, # choose product
province_id = 50, # choose province
`member_type[]` = 1, # agency level 1
`member_type[]` = 2 # agency level 2
)
)
data <- httr::content(pg$response)$data
结果:
推荐阅读
- python - 熊猫系列滚弦
- jenkins - Webhooks 如何让 BRANCH_NAME 由 Jenkins 上传
- scala - 不能在 intellij 中包含 org.apache.spark
- core-graphics - “将图形上下文保持在任何状态”的含义
- r - 匹配 gsub 模式中的下一个符号
- c# - C# 十六进制到字节数组循环
- tcl - 配置:错误:在 /home/dave/src/tcl-builds/raw/tcl8.4.1 中找不到私有头文件 tclInt.h
- android-graphview - 找不到 LineGraphView、GraphViewSeries、GraphViewData
- r - 如何根据 R 中的集群分离数据
- facebook - Facebook 应用帖子不会向用户显示